World: r3wp

oldes... no I didn't know this.

but I never let view load images on its own, I always control that. 
 I didn't even know about load-image  ;-)

henrik, if a face isn't unset properly, for sure the image will stay 
stuck in ram... since one reference to it exists.

BUT


I've previously realised that using the stylesheet system actually 
runs init on each face when you build the stylesheet.  as the stylesheet 
is linked within the faces, each stylesheet will effectively constitute 
a memory reference on some resources.

oldes, about the load ... its not totally obvious why sometimes the 
images stay stuck in ram... but in this case it was huge png images. 
 and I did every recycle trick I new of... I tried solving this for 
2 hours... but nothing worked... not even allocating other stuff, 
and freeing it, to make sure the GC really did do a cleanup (in case 
is was count based).

now that I think about it... its not loading the images, it was using 
make image which caused the memory leak... if you call 400MB a "leak".

this is for a pretty insane project I have which will use up about 
30-40GB of image data (once loaded pictures take up much more space 
than on disk, obviously), per output picture.

Maybe it's the content of PNG images what couses problems with GC. 
What do you do with so many images?

I use ImageMagick as well. It allows much more than pure Rebol. But 
of course I call it from Rebol:)

hehe I did the same :-)  I have an automated picture catalogue builder...

its part of a huge visual arts project I have which requires thousands 
of photos... all in high quality... the end goal is a 1.75 x 6m wide 
printout. which use many overlaid independent images.

by many, I expect at least 10000.

Interesting:)

output res should be at 600dpi.  the trick is to find a print shop 
which has a large-print printer with enough ram in its module... 
most shops top out at 2-3m

but i need to build the software first, and right now.. that depends 
(again) on R3... I will be using OpenGL to build the output image 
setups, then dumping the info to a monstrous image magic process, 
which in the end might actually process more raw image data than 
my HD actually can contain.  I expect render times in number of days. 
 (4 pictures on a much smaller canvas took more than an hour).

the processing will be spread out on a farm and reintegrated, as 
tiles render-out.

the subjects of the system can change at will, this whole system 
is a process, not a specific one.  in the best of worlds, I'd find 
a philantropist that would allow me to build the full project (at 
a much larger scale), which costs quite a few thousand dollars in 
printing... one day...

but the technology, once built... can be reused for any project  
 at any scale.

what is the source of images?

the current subject is a church next to my house. I already have 
around 3000 pictures of it at various periods... but the real project 
is a 24hour shooting session, with a top-of the line rented digital 
cam.

I'd like to use hdri imagery.  but with image size that can reach 
in exces of 100mb per picture... its a bit prohibitive on the amount 
of pictures you can take before offloading.

so actual details will vary on time and budget when the final step 
of the project is green-lit... not this year for sure.

like any serious art project... it takes time to build up the hype 
around it too, and possibly get subsidies to pay for part of it.

I need to pay the RAID tower, the hours I'll pour on programming, 
photography, img tagging and output image synthesis qualities, etc. 
 publicity and find a few places to show off the end-results... which 
sometimes has to be paid in advance...  


life is so simple when you have no interests... sometimes I'd like 
to be that way for just a few days a year... hehehe.

Sunanda: if an app uses a huge peak at startup and then does not 
use that memory anymore, that memory just gets swapped to disk. So, 
it's not really in the way of other applications.

that still slows things down, doesn't it?

It may not seem obvious, but 

	view layout [image %file.png]

does make use of LOAD-IMAGE, as Oldes pointed out.
Here is where the IMAGE style does it:

	print mold get in svv/vid-styles/image/multi 'file

Gabriele: that may be true in an ideal world, but not in a Windows 
one.


I tried the test code below in R2 on a freshly booted machine.....I'd 
start a REBOL session, paste the code in,ánd wait until it gave me 
the timing. Then start another session (leaving the old one active) 
and repeat.


I only needed five console sessions to exhaust all physical memory, 
and have Windows behaving in an unstable manner. Killing the console 
sessions returned things to almost normal.


Basically, if this code modeled an application's start-up memory 
usage, it would be unwise to run it under Windows:

rebol []
    t: now/time/precise
    b: copy ["a"]
    attempt [
        forever [
          append b b
          append b join last b last b
          ]
     ]
     print length? b
     foreach a b [clear a]
     clear b
     recycle
     print now/time/precise - t
     halt

Is it REBOL's fault that Windows sucks? ;)

Maybe Carl should treat Windows as a special case...

but I've been told before that rebol doesn't deallocate ram, it only 
frees it from the pool.

so this might not just be a windows issue.

and using swap still takes up ram, it prevents other apps from using 
swap, so its a bad idea on any platform to occupy several hundred 
megs of unneeded ram.

on windows, the detail is that even the OS uses swapable RAM by default. 
 there is registry setting to prevent this, and it makes a hell of 
a big difference in windows itself.  when the machine is under load, 
the OS stays responsive, as opposed to becoming a hog.

I remember doing 3D rendering where up to 4GB of swap can be needed, 
just to get the scenes to render... had I known about this switch 
then, It would have made such a difference.  waiting 30 seconds for 
the task manager to open, cause it has to free it from swap is just 
not feasible when you are charging clients per hour.

In Windows you want virtual memory to be turned on, but you can set 
it to not use a swap file if you like, without touching the registry. 
Virtual memory is used to support memory-mapped files and speeds 
program loading and use (which is done with memory-mapping). The 
computer I am on right now has virtual memory and no swap file since 
it has an SSD, and it runs fine.

You better be sure you have enough RAM though, and for some purposes 
you might need 64bit Windows to even support enough RAM.

In my experience, even with 2GB of ram on a machine with xp and one 
or two software, you will get OS out of memory errors if you turn 
off swap files.  this, with 1GB physical RAM still available... I've 
tried it a few times before and after a short period I had to reboot 
my machine.   The kernel is built in such a way that it expects it 
and it just can't really cope without it.

just to get the scenes to render
 - this might explain your experience :)

the registry tells the OS not to use ram which can be swapped to 
disk, eating up about 500MB.  its still virtual, it just doesn't 
swap to files.

no that is different usage... hehe

the registry *hack*.

I get out-of-memory errors sometimes even with a swap file :)

increase it and it will disapear... and don't let the OS auto-increase 
the size.... that leads to other out-of memory conditions when your 
disks are near full.

However, I tend to not get out-of-memory errors on this computer, 
which has 1GB of RAM and no swap file. This is because I am careful 
about which programs I use (no IE, no Firefox).

has anyone been successful in compressing data in php and decompressing 
it again in R2? I see I asked such a question 7 years ago on the 
mailing list, but no solution was found back then. :-)

I think I've figured it out. Anyhow, the solution I have working 
here seems stable.

using ZLIB?

I was nagging you about the actors+message passing rebol lib I was 
playing with..  Now I have a real thing running with it .. I will 
post source code on blog soon. It's not that exciting video .. just 
some text changing :) but I hope it will help in getting a picture 
when I post the code.


On left is a Linux server with work dispatcher on  right are two 
workers on windows. There can be as many workers on many computers 
or none and workers can drop out at any time without work being undone/"halfdone". 


It uses message passing and actors for everything.. and comunication 
works like at tuple spaces so you get autobalancing. It's not something 
that special, you could do something roughly the same with http server 
app probably, although it would be a little more messy.

video is here: http://screencast.com/t/ieXjmEz2b9:)

I needed this setup because I need to make screenshots of websites 
in browsers for websites in site assistant and you can't make a screenshot 
of IE browser on linux. I don't intend to have any windows servers 
running but I will have workers on mine and computers of colegues 
running on background and doing it's job silently. Thats why it's 
all made so that there is nothing wrong even if no workers are avalable 
for bigger amounts of time

and they can get on and off work as they please :) .. I call this 
"casual workers system" :)