AltME groups: search

Diffferent, but related......memory may be deallocated within REBOL's 
sphere of influence one the items have no more references; but it 
is not (it seems) dellocated back to the opsys until the REBOL thread 
closes.


That can create several problems ..... eg an application that uses 
a huge peak memory load at startup, and would like to hand it back 
for the next 8 hours of running. It's all in the arena of efficient 
memory management.

There is a difference between loading image using 'load function 
and loading using 'load-image function (which is used by VID). The 
second one stares data in cache.

henrik, if a face isn't unset properly, for sure the image will stay 
stuck in ram... since one reference to it exists.

BUT


I've previously realised that using the stylesheet system actually 
runs init on each face when you build the stylesheet.  as the stylesheet 
is linked within the faces, each stylesheet will effectively constitute 
a memory reference on some resources.

oldes, about the load ... its not totally obvious why sometimes the 
images stay stuck in ram... but in this case it was huge png images. 
 and I did every recycle trick I new of... I tried solving this for 
2 hours... but nothing worked... not even allocating other stuff, 
and freeing it, to make sure the GC really did do a cleanup (in case 
is was count based).

now that I think about it... its not loading the images, it was using 
make image which caused the memory leak... if you call 400MB a "leak".

this is for a pretty insane project I have which will use up about 
30-40GB of image data (once loaded pictures take up much more space 
than on disk, obviously), per output picture.

its part of a huge visual arts project I have which requires thousands 
of photos... all in high quality... the end goal is a 1.75 x 6m wide 
printout. which use many overlaid independent images.

output res should be at 600dpi.  the trick is to find a print shop 
which has a large-print printer with enough ram in its module... 
most shops top out at 2-3m

but i need to build the software first, and right now.. that depends 
(again) on R3... I will be using OpenGL to build the output image 
setups, then dumping the info to a monstrous image magic process, 
which in the end might actually process more raw image data than 
my HD actually can contain.  I expect render times in number of days. 
 (4 pictures on a much smaller canvas took more than an hour).

the processing will be spread out on a farm and reintegrated, as 
tiles render-out.

the subjects of the system can change at will, this whole system 
is a process, not a specific one.  in the best of worlds, I'd find 
a philantropist that would allow me to build the full project (at 
a much larger scale), which costs quite a few thousand dollars in 
printing... one day...

the current subject is a church next to my house. I already have 
around 3000 pictures of it at various periods... but the real project 
is a 24hour shooting session, with a top-of the line rented digital 
cam.

I'd like to use hdri imagery.  but with image size that can reach 
in exces of 100mb per picture... its a bit prohibitive on the amount 
of pictures you can take before offloading.

I need to pay the RAID tower, the hours I'll pour on programming, 
photography, img tagging and output image synthesis qualities, etc. 
 publicity and find a few places to show off the end-results... which 
sometimes has to be paid in advance...  


life is so simple when you have no interests... sometimes I'd like 
to be that way for just a few days a year... hehehe.

Sunanda: if an app uses a huge peak at startup and then does not 
use that memory anymore, that memory just gets swapped to disk. So, 
it's not really in the way of other applications.

Gabriele: that may be true in an ideal world, but not in a Windows 
one.


I tried the test code below in R2 on a freshly booted machine.....I'd 
start a REBOL session, paste the code in,ánd wait until it gave me 
the timing. Then start another session (leaving the old one active) 
and repeat.


I only needed five console sessions to exhaust all physical memory, 
and have Windows behaving in an unstable manner. Killing the console 
sessions returned things to almost normal.


Basically, if this code modeled an application's start-up memory 
usage, it would be unwise to run it under Windows:

rebol []
    t: now/time/precise
    b: copy ["a"]
    attempt [
        forever [
          append b b
          append b join last b last b
          ]
     ]
     print length? b
     foreach a b [clear a]
     clear b
     recycle
     print now/time/precise - t
     halt

Maybe Carl should treat Windows as a special case...

so this might not just be a windows issue.

and using swap still takes up ram, it prevents other apps from using 
swap, so its a bad idea on any platform to occupy several hundred 
megs of unneeded ram.

on windows, the detail is that even the OS uses swapable RAM by default. 
 there is registry setting to prevent this, and it makes a hell of 
a big difference in windows itself.  when the machine is under load, 
the OS stays responsive, as opposed to becoming a hog.

I remember doing 3D rendering where up to 4GB of swap can be needed, 
just to get the scenes to render... had I known about this switch 
then, It would have made such a difference.  waiting 30 seconds for 
the task manager to open, cause it has to free it from swap is just 
not feasible when you are charging clients per hour.

In Windows you want virtual memory to be turned on, but you can set 
it to not use a swap file if you like, without touching the registry. 
Virtual memory is used to support memory-mapped files and speeds 
program loading and use (which is done with memory-mapping). The 
computer I am on right now has virtual memory and no swap file since 
it has an SSD, and it runs fine.

In my experience, even with 2GB of ram on a machine with xp and one 
or two software, you will get OS out of memory errors if you turn 
off swap files.  this, with 1GB physical RAM still available... I've 
tried it a few times before and after a short period I had to reboot 
my machine.   The kernel is built in such a way that it expects it 
and it just can't really cope without it.

I get out-of-memory errors sometimes even with a swap file :)

has anyone been successful in compressing data in php and decompressing 
it again in R2? I see I asked such a question 7 years ago on the 
mailing list, but no solution was found back then. :-)

I was nagging you about the actors+message passing rebol lib I was 
playing with..  Now I have a real thing running with it .. I will 
post source code on blog soon. It's not that exciting video .. just 
some text changing :) but I hope it will help in getting a picture 
when I post the code.


On left is a Linux server with work dispatcher on  right are two 
workers on windows. There can be as many workers on many computers 
or none and workers can drop out at any time without work being undone/"halfdone". 


It uses message passing and actors for everything.. and comunication 
works like at tuple spaces so you get autobalancing. It's not something 
that special, you could do something roughly the same with http server 
app probably, although it would be a little more messy.

I needed this setup because I need to make screenshots of websites 
in browsers for websites in site assistant and you can't make a screenshot 
of IE browser on linux. I don't intend to have any windows servers 
running but I will have workers on mine and computers of colegues 
running on background and doing it's job silently. Thats why it's 
all made so that there is nothing wrong even if no workers are avalable 
for bigger amounts of time

I haven't seen it but heard of doc's task manager, and he uses multiple 
worker processes in cheyenne. I am certain those two (or one if it's 
the same) are much faster and robust. I am mostly experimenting here 
on how a "message passing with actors" system would or could look 
in REBOL , so I am not focused on implementation, but for now more 
on look of code.

two processes ping ponging to eachother 10 times with actor-net would 
look something like this now

pinger: make actor [ act-match: [ 

    [ 'pong count addr ] [ ~send addr compose [ ping (count) (get-my-addr) 
    ] ] 
    [ ''bye ] [ print "ok, bye" ]
]
ponger: make actor [ act-match: [ [ 'ping count addr ] [ 

        ~send addr compose ( either lesser? count 10 [ [ pong (count) (get-my-addr) 
        ] ] [ [ bye ] ] )
    ] 
]


and you would add the third actor that would act only once to "lend 
them the ball" to start them


server: make once-actor [ act-once: [ ~send get-actor 1 compose [ 
ping 0 (get-actor 2) ] ] ]


But the library is still quite messy .. especially with adresses 
of actors, global vars, I have a big list of things now to improve 
after I made that thing on the screencast

I need to view various files in a browser and if I use 'browse, then 
this blocks the UI until the browser appears with the file.  I just 
tried with a xmlrpc server accepting the url and it works fine.

how do you get a return value .. with xmlrpc-exe ... what do you 
mean a page pops up?

I have to admit I don't get what borwser (you mean like IE FF?) has 
to do with xmlrpc in this case ... and I.. aha this is a GUI app 
you you don't mean that xmlrpc-exec returns imediately (without waiting 
for result -- thats why I ask how you get the resut then) .. but 
that GUI doesn't block while waiting for it

This is pretty cool ... I am sitting in front of my laptop, and on 
the other side of the room is my 46" LCD screen with my media PC 
attached.  I installed the rebxr xmlrpc server on it .., and I have 
cheyenne installed on the laptop.  I click on a file in my application, 
it gets downloaded to the www directory in cheyenne, and then I send 
a command to the xmlrpc server to browse to that file so that it 
displays on the big screen but is served up by the laptop's cheyenne 
server.

It's now quite remote controlling a browser on another PC because 
all I can do is tell it to open a new page.

I presume using xmlrpc I can ask the server to spawn more instances 
of itself on different ports so that I cah have a number of different 
"tasks" available

Question : 
I have a script like this :

ctx: context [
	markup: "my script with <%val1%> and <%val2%>"
	val1: "Hello"
	val2: "Bye"
	build: does [build-markup markup]
]

It produce this result :
>> ctx/build

== {my script with ***ERROR no-value in: val1 and ***ERROR no-value 
in: val2}


Its because 'build-markup use 'parse and 'parse looks its word in 
the global context.

Is there any binding tricks to solve this ?

Smart Guys... What's the trick to using a variable to hold an email 
address when using "Send"
Send/header  some-email-address Some-text myheader
I always get a failed email because  the "To:"  word is not set.
If you directly type in an email address such as
Send/header  [confused-:-rebol-:-com] Some-text myheader

It works. I've tried composing, reducing, scrubbing and washing and 
the stains still don't come out, if you know what I mean.

There must be some trick. (And I know you know)

Its weird. I've tried placing it in a block, etc.

then it would be a bug, if it doesn't get passed to 'send.

just had a look at the SEND source and from what I saw it is highly 
unlikely that it does not work Moreover, I am actually using the 
SEND function from time to time and no such problem occurrred.

Henrik and Ladislav, 

OK, stupid me. I was using a word name that I shouldn't have. Thanks!

Didier, assuming you can't preface the markup fields with the context 
name, e.g. 

  markup: "my script with <%ctx/val1%> and <%ctx/val2%>"


either manually or programmatically, hacking build-markup may be 
best. e.g., add a /with refinement that takes the object and bind 
to that.

I wrote a blogpost with code about the usage of actor-net lib that 
we chatted with Graham a little (few days back) http://itmmetelko.com/blog/2009/07/03/rebols-actor-net-used-for-real-distributed-system/

I suspect a lot of rebol readers won't be fully conversant with Erlang.

who said he was going to implement a distributed Erlang system in 
Rebol .... some time ago !

(I don't know a lot about it, only it seemed minimal and clear in 
concept which I liked)

janko, maybe you should look into liquid a bit... maybe you'd have 
some perspective on how its kernel could embed an actor... then you'd 
have a robust management layer with very powerfull computing models 
all integrated into one system.  using actors or something similar, 
maybe we could allow the processes themselves to be offloaded.

this is something I wanted to build a long time ago, but just now 
seem to be needing, with remark development.  the ability to break 
up a page's rendering over a network of machines could be quite powerfull 
in allowing extremely high-loads.

very cool ! .. because none of you gurus ever responded to my 3 posts 
about actor-net I was begining to think I am pushing all this into 
totally stupid direction and I don't even see it (I am not CS educated) 
, now that I know I can chase this with a little more certanty .. 
so this is definately a positive push for me

This has been a really busy week for R3 semantics and changes - don't 
take it personally.

no problem at all .. I am very stubborn so I doubted in this only 
at a moment of weakness :)

yes, when I started it I was thinking that it could serve as some 
food for thought on R3 concur model .. because Pekr and others mentioned 
erlang a lot, so I wanted to see how erlang would look in rebol at 
all

Erlang has been getting a lot of attention lately because it has 
a concurrency model that doesn't suck, which is amazing relative 
to most other systems. However, it's syntax is not great - we can 
do better :)

and it has slow IO .. slower than even other dynamic langs .. another 
minus of it is it's limitation at strings .. so people use binaries 
for strings so code looks like this <<"somestring">> + <<"sadasd">> 
.. quite horrible .. if it werent for these strings I would use it 
for something a while back but strings are the base of webapps and 
writing <<"">> all the time seems horrible

I've been looking at ropes lately - it's a kind of functional string 
structure built on trees of immutable substrings. Someone did an 
article about it recently. I think we can implement this in R3 as 
a user-defined string type. Ropes could be useful for increasing 
performance and lowering memory overhead for R3 multitasking and 
large string processing.

I read the article linked from reddit. The author's opinions were 
a bit off sometimes, but the principle is sound. Combining ropes 
with the copy-on-write idea gets you copy-on-write strings that don't 
have to really copy if they don't need to. Only the modified part 
would need new storage - the unmodified part could be referenced 
unchanged. When you combine this with the possibility that PROTECTed 
series may be thread-global in R3, but unprotected series thread-local, 
then functional data structures start to become interesting in R3.

hm.. this would be very good to have .. and it sounds very much like 
data structures that Clojure has ... I think it calls them persistend 
data structures , like list, map, etc .. and the same as you described 
here. You have many versions of a data structure but they share the 
unchanged parts.. this highly benefits functional programming style 
.. you get "new" data structure each time, but without the penalty 
of copying it each time , and is cruicial in concurrency which is 
in focus in clojure, because then different threads dont's share 
and "corrupt" one to another the data structure , but each can have 
it's own revision , etc .. 


I am not that good in this complex stuff, but it might be very intersting 
to you becuase you will know much better what's he talking about 
and how can that be implemented .. they made some of his data structures 
in Factor , so they ren't impossible to make in other languages.. 
I don't remember exacty where I read about this

This is where persistent data structures come in. Based on the work 
of Phil Bagwell, Rich Hickey described how he used bit partitioned 
hash tries to make efficient "copies" of data structures, and this 
forms the basis of data storage within Clojure. Essentially all data 
is stored in a tree and when one makes a copy with a small change, 
one can create a tree with a new root and only the path to the changed 
item needs to be copied and modified. The rest of the tree's branches 
remain precisely the same. This significantly reduces the amount 
of copying that is required and makes multiple "versions" of a data 
structure entirely practical.

<<< that upthere is a excert from that article

and if you want to do for example efficient message passing concur. 
it's also cruicial to have this sort of data.. because message is 
always a copy .. if you are using message passing for distr. compuring 
- only between computers then you have to copy anyway so it's no 
penalty .. but if you use message passing for concurrency /paralel 
execution on one computer then copying data for messages each time 
will have a high penalty , but it wouldn't with ropes for example

To use it, double click on the blank canvas to create nodes, and 
double click on nodes to edit them.


Drag arrows from one node to another by click and drag from the left 
bottom corner.

Control-L to load a new map

Control-S to save a new map

Brian, yes.. since I like the FP a lot immutable stuff is certanly 
what I like , I sometimes discover looking at things this way solves 
or uncomplicates a lot of traditional imperative problems

Janko, how about attempting a CORBA interface ... that would mesh 
in with what you're doing

Yes, that's a known issue.

Within an async handler things might get unpredictable. If you have 
a normal event loop, including async in the wait-list, I think you 
should be good.

I was wondering how to download a 120Mb file from S3

Anyone have a nifty func to "untrim" a string? Something that converts:

	"ArialBold"

to:

	"Arial Bold"

Best I can come up with is a horrible replace loop:

	foreach char "ABC ..." [
		replace/all next string char join " " char
	]

 replace/all string "  " " " ; handle case where words were already 
 space seperated

caps: charset [ #"A" - #"Z" ]
non-caps: complement caps

parse "ArialBold" [ some [ copy fontname name (print fontname)  ]]

not sure if correct, but:

caps: charset [ #"A" - #"Z" ]
non-caps: complement caps
s: "aaaArialBoldItalic"

parse/all s [any [mark: caps (insert mark #" ") skip | skip] end]

'some works. The 4 variations it has to handle are:

	"Arial"
	"ArialBold"
	"ArialBoldItalic"
	"ArialCY"

so putting a 'next prior to 's seems to handle all these.

why is it a hack??

Here's the finished code (which obtains REBOL compatable font names 
under Mac):

	fonts: copy []
	caps: make bitset! [#"A" - #"Z"]

 foreach file compose [(read %/System/Library/Fonts/) (read %/Library/Fonts/)] 
 [
		if %.dfont = suffix? file [
			s: form first parse file "."

   parse next s [any [mark: caps (insert mark #" ") 2 skip | skip] end]
			insert tail fonts s
		]
	]
	remove-each font-name fonts: sort unique fonts [

  (size-text make face [text: "A" font: make face/font [name: font-name 
  size: 10]]) =

  size-text make face [text: "A" font: make face/font [name: font-name 
  size: 12]]
	]


(the windows func to do this is http://www.reboltech.com/library/scripts/get-fonts-windows.r
)

there's no edge condition. What if the name would not begin with 
a capital letter? I would parse all string and instead of 'next I 
would use 'trim, which would tream initial space, in case first letter 
is capital :-) But if it is the rule, that the first letter is always 
being a capital, then your solution is absolutly correct ...

First letter is always a capital ... except where it's a # but those 
fonts are not usable by REBOL anyway.

What's the best way of stopping an application from starting if another 
instance is already running?  The app is a core based app that is 
in a forever loop.

I guess I could always make it a View app instead, and wait on  a 
port ... and use a timer to trigger the other stuff I need to do.

so, open a file and keep it open?

open a listen port, there can only one be open

One way:

On startup:
-- check for your timestamp file

-- If it does not exist or  (it exists and timestamp is over 2 minutes 
in past), proceed to run

-- otherwise, wait 65 seconds.  Test if timestamp has changed: yes-halt; 
no-proceed

While running:

-- write the  timestamp file at least once a minute with an updated 
time

On clean closedown:
-- delete the timestamp file.

Drawbacks:

-- application could take over a minute to restart if immediately 
restarted after a crash.

-- manual deletion of timestamp file can lead to multiple instances 
running (you can minimise this by re-reading file and aborting if 
timestamp is not the last one you set)
-- all those writes of the file.

Sunanda,  does your timestamp file mean a file with the timestamp 
as content or just the date and time of the file?

I have many times seen, that the timestamp of a file under windows 
does not change, although there is always data added to the file.

sqlab -- I mean a file whose contents is just a timestamp, eg:
  write %timestamp.txt mod now/precise

In practice, the actual method is a little more convoluted (to avoid, 
for example, two instances both starting at once -- so neither sees 
a pre-existing timestamp.txt file).

I need to issue the combination ALT+RETURN into a string. How can 
I do this? Is there a control character for ALT like for CTRL?

I tried various HEX codes I found in google but none worked. Excel 
just printed strange chars instead of doing a line-break.

is alt+return really a char?

Robert, you're mentioning Excel, so you're probably trying to do 
ALT+RETURN hard line-breaks with Excel thru comlib?

Maybe you can use the actual character code that Excel uses whenever 
one presses Alt+Enter. The web says Excel just inserts a line feed 
#"^/", have you tried that?

Alt+RETURN is not a character

Christian, right, I try to insert a hard line-break in Excel. I tried 
inserting NEWLINE, which doesn't work. Will try just the LN. Thanks.

Apart from chucking an error, what is the best way to determine if 
a path exists inside an object?  So, the path might be 10 deep ...

As far as I know, Graham, there is not a simpler way than provoking 
an error.
If you are already at the bottom of the structure FIND can help:

    a: make object! [b: make object! [c: 1]]
    find next first a/b 'd     ;; is a/b/d valid?
    == none
    find next first a/b 'c     ;; is a.b.c valid?
   == [c]


But recursing to the bottom may be slower than just trying the error?:
   error? try [a/b/d]
   error? try [a/b/c]

It's a poorly specified way of obtaining some information.

perhaps there is merit for a feature to quickly determine the existence 
of a deeply stored value?

because 'in expects a word

And why is that such a problem?

I mean, it's a solvable problem. IN could theoretically be enhanced 
to also accept paths, but, iIrc, Carl wasn't keen on the idea because 
he wanted IN to remain simple and fast. He said something like "it's 
for words".

(Ok, it's not a solvable problem for paths with dynamic elements 
in them like functions, but of course, there are many more "static" 
paths which this would be darned useful for.)

So you have to check for 'to in object/path first, and so on. I didn't 
say it would be a fast mezzanine :(

I guess there's a lot of advantages to just using blocks ... as it 
is now, some elements in my object are blocks and some are objects 
which makes it very messy

OTOH, blocks could get messy too ...


at present I have structures like ccr/body/alerts/alert where alert 
can be either a single alert object, or a block of alert objects

Ideally if I were to use blocks, it would end up that alert is always 
a block .. 0 ... n blocks

The R3 GUI is structured as a mix of objects, maps, blocks and gobs, 
but it is very consistent and not messy.

I presume it can't change though ... if a particular item is an object!, 
it won't be a block of objects in another instance ....