Mailing List Archive: 49091 messages
  • Home
  • Script library
  • AltME Archive
  • Mailing list
  • Articles Index
  • Site search
 

A collection of garbage

 [1/19] from: patrick::philipot::laposte::net at: 8-Jul-2003 13:46


Hi List I often code functions returning a block. What happens to all these blocks? Are they "garbage collected"? I doubt it ... Imagine a code like this one: gimme: func[n][b: copy [] for i 1 n 1 [append b i] b] print gimme 5 print gimme 15 print gimme 10 As I see it, there are 3 blocks that will stay in memory forever. And this slightly different version seems not better: gimme: func[n /local b][b: copy [] for i 1 n 1 [append b i] b] The best I can think of is to provide a block to the function: gimme: func[n b [block!]][clear b for i 1 n 1 [append b i] b] result: copy [] print gimme 5 result print gimme 15 result print gimme 10 result Any inspired comments? Regards Patrick Regards Patrick

 [2/19] from: g:santilli:tiscalinet:it at: 8-Jul-2003 14:44


Hi Patrick, On Tuesday, July 8, 2003, 1:46:17 PM, you wrote: p> What happens to all these blocks? p> Are they "garbage collected"? I doubt it ... They will, if they are no more referenced. Regards, Gabriele. -- Gabriele Santilli <[g--santilli--tiscalinet--it]> -- REBOL Programmer Amigan -- AGI L'Aquila -- REB: http://web.tiscali.it/rebol/index.r

 [3/19] from: antonr:iinet:au at: 9-Jul-2003 0:29


Well, we could probably check whether unreferenced blocks are recycled by using the new 'stats function:
>> ? stats
USAGE: STATS /pools /types /series /frames /recycle DESCRIPTION: System statistics. Default is to return total memory used. STATS is a native value. REFINEMENTS: /pools -- Returns: width units free-units units-per-alloc segments mem-in-use /types -- Returns: Datatype count /series -- Returns: total blocks strings other avail free expansions /frames -- Returns: total used unused free values-total /recycle -- Returns: count series-total series-last frames-total frames-last ballast Anton.

 [4/19] from: patrick:philipot:laposte at: 8-Jul-2003 17:09


Hi Gabrielle, As Fox Mulder would say "I want to believe". But it's a very short answer to a very big problem ! Has anyone more on this subject? Regards Patrick

 [5/19] from: maximo:meteorstudios at: 8-Jul-2003 13:27


excuse me for butting in ;-) Iconify a view app, and you will see that its memory consumption will drop ! you can see this with the system allocation tool. I am not sure if its rebol that's de-allocating its ram or if its windows that hidding that app's deep in cache files... In a typical test My app goes from 30MB to ~4-5MB and then back to 30MB when its de-iconified... Just adding a little mystery to the MEM issue... ;-) has anyone tested if any memory is freed when a call to clear is done ? -max ----------- meteor Studios, T.D. ----------- Strong enough for a man, but made for a woman

 [6/19] from: rotenca:telvia:it at: 8-Jul-2003 19:12


> But it's a very short answer to a very big problem ! Has anyone more on this > subject?
1) I am not sure that Rebol releases allocated memory to the system(i think it never happens). Use system/stats to check ebol memory allocation. 2) I am sure that Rebol re-use its no more referenced memory. You can make some tests about this with http://www.reboltech.com/library/scripts/mem2.r --- Ciao Romano

 [7/19] from: maximo:meteorstudios at: 8-Jul-2003 14:38


I read my mail and I just wanted to precise that the "system allocation tool", really refers to "Windows Task Manager"... sorry for the bad wording...

 [8/19] from: rotenca:telvia:it at: 8-Jul-2003 20:40


> 1) I am not sure that Rebol releases allocated memory to the system(i think
it
> never happens).
I was wrong. It releases some memory under some conditions. But with an hard to predict logic. --- Ciao Romano

 [9/19] from: nitsch-lists:netcologne at: 8-Jul-2003 21:21


Am Dienstag, 8. Juli 2003 19:27 schrieb Maxim Olivier-Adlhoch:
> excuse me for butting in ;-) > > Iconify a view app, and you will see that its memory consumption will drop > ! you can see this with the system allocation tool. > > I am not sure if its rebol that's de-allocating its ram or if its windows > that hidding that app's deep in cache files... In a typical test My app > goes from 30MB to ~4-5MB and then back to 30MB when its de-iconified... >
You could run a test with /view and /core. I expect they comparable in size when /view is iconified. It holds a lot of 24-bit double-buffers AFAIK
> Just adding a little mystery to the MEM issue... ;-) >
-Volker

 [10/19] from: nitsch-lists:netcologne at: 8-Jul-2003 21:19


Am Dienstag, 8. Juli 2003 17:09 schrieb patrick:
> Hi Gabrielle, > > As Fox Mulder would say "I want to believe". > > But it's a very short answer to a very big problem ! Has anyone more on > this subject? >
If its a very big problem, Gabriele would have noticed it and published workarounds. So its collected :) rebol waits until there is some stuff to collect, usually some MB. Then it collects a lot of stuff at once. Like people do with garbage ;)
> Regards > Patrick
<<quoted lines omitted: 18>>
> > Amigan -- AGI L'Aquila -- REB: http://web.tiscali.it/rebol/index.r > >
-Volker

 [11/19] from: carl:rebol at: 8-Jul-2003 20:06


The logic is fairly complicated, but it's also quite fast. For example, you might be using 50MB of memory, but not even notice when the GC happens. There are many REBOL programs that run for months (years?) at a time without restarting, so we know the GC is stable. (There are a few exceptions, in operations like TRY on OPEN failures, etc.) If you're curious about your code's memory, just put your code into a loop and print memory stats every so often. (Use a newer version of REBOL for better accuracy on stats.) loop 10000 [ your code print ["Mem:" stats] ] You'll see the mem usage top out after a while (unless you are doing things like continuously appending to series, such as when keeping a log). The two basic memory methods in REBOL are: * Small allocations (values, blocks, etc.) are managed by REBOL for speed reasons. They are efficiently recycled. * Large allocations (files, images, sounds, big blocks, etc.) are allocated thru the native OS, because we figure it knows better the characteristics of its VM paging model and can optimize the allocation. So, here's what programmers need to remember: * If you're just using normal REBOL values and small series, just sit back and relax. * If you allocate large memory objects (strings, images, blocks, etc.), you may want to un-reference them when you are done with them. For example: file: read %large-file.txt do some-stuff-on-file file: none ; allow file buffer to GC eventually This is not required, but it will help reduce memory usage. IMPORTANT: in the example above, if the file variable is a local variable to a function, DO NOT ASSUME that it will be un-referenced automatically. By default, the first frame (first level of call) of local variables for a function remain referenced. For example, if you have a function: foo: has [file] [ file: read %large-file.txt print length? file ] The file memory will not be released until the local variable is set to a different value (by a second call to foo, which will then hold the file memory until the next call, etc.) To change that, unreference the file series when done: foo: has [file] [ file: read %large-file.txt print length? file file: none ] -Carl

 [12/19] from: fsievert:uos at: 9-Jul-2003 10:22


Hi Carl, there is still a strange thing with nested foreach, do/next and reduce's. test: func [:looper x] [ if x < 100 [looper z [1] [ test looper x + 1 prin "." ]]] [test repeat 1] and [test foreach 1] are doing exactly the same thing, but loop 22 [test repeat 1] works, while loop 22 [test foreach 1] crashes. Turn recycler off to see both working. You can see the same problem with do/next and reduce. Frank On Tue, 8 Jul 2003, Carl at REBOL wrote:

 [13/19] from: cyphre:seznam:cz at: 10-Jul-2003 9:04


Hi Carl, Frank and all,
> ----- Original Message ----- > From: "Frank Sievertsen" <[fsievert--uos--de]>
<<quoted lines omitted: 6>>
> Frank > ......
What about this? Just try it to paste into console with and without commented "recycle/off" line: -----------------begginning of code------------------- address?: function [ {get the address of a string} s [any-string!] ] [address] [ s: make struct! [s [string!]] reduce [s] address: make struct! [i [integer!]] none change third address third s address/i ] get-mem?: function [ {get the byte from a memory address} address [integer!] /nts {a null-terminated string} /part {a binary with a specified length} length [integer!] ] [m] [ address: make struct! [i [integer!]] reduce [address] if nts [ m: make struct! [s [string!]] none change third m third address return m/s ] if part [ m: head insert/dup copy [] [. [char!]] length m: make struct! compose/deep [bin [struct! (reduce [m])]] none change third m third address return to string! third m/bin ] m: make struct! [c [struct! [chr [char!]]]] none change third m third address m/c/chr ] img: make image! 512x512 ;recycle/off ;Warning! should allocate about 200MB of memory but don't crash loop 4 [ get-mem?/part address? img length? img ] ----------------------------------end of code --------------------------------- Looks like there are still problems with the GC?? regards, Cyphre

 [14/19] from: lmecir:mbox:vol:cz at: 10-Jul-2003 12:48


Hi Cyphre and Gabriele, I changed the GET-MEM? function implementation slightly to use a "truly dynamic" local variable. I found out, that the crash occurs even if we don't use the cycle: address?: function [ {get the address of a string} s [any-string!] ] [address] [ s: make struct! [s [string!]] reduce [s] address: make struct! [i [integer!]] none change third address third s address/i ] get-mem?: func [ {get the byte from a memory address} address [integer!] /nts {a null-terminated string} /part {a binary with a specified length} length [integer!] ] [ use [m] [ address: make struct! [i [integer!]] reduce [address] if nts [ m: make struct! [s [string!]] none change third m third address return m/s ] if part [ m: head insert/dup copy [] [. [char!]] length m: make struct! compose/deep [bin [struct! (reduce [m])]] none change third m third address return first reduce [third m/bin m: none] ] m: make struct! [c [struct! [chr [char!]]]] none change third m third address m/c/chr ] ] img: make image! 512x512 recycle/off ;Warning! should allocate about xxxMB of memory get-mem?/part probe address? img length? img get-mem?/part probe address? img length? img recycle get-mem?/part probe address? img length? img -L

 [15/19] from: patrick:philipot:laposte at: 12-Jul-2003 11:15


Hi List Stats/types fails when used just after starting Rebol. REBOL/View 1.2.10.3.1 30-May-2003 Copyright 2000-2002 REBOL Technologies. All rights reserved. ... License ID: 1-01102-240
>> print stats/types
Stats: improper block termination: #6D00A0 Stats: improper block termination: #6D00B0 Stats: improper block termination: #6D00C0 Stats: improper block termination: #6D00E0 Stats: improper block termination: #6D0160 ** Script Error: end! word has no context ** Near: end! 377 unset! 3454 error!
>>
Regards Patrick

 [16/19] from: Carl:rebol at: 12-Jul-2003 2:43


Hi Patrick, About:
>> print stats/types Stats: improper block termination: #6D00A0
Stats: improper block termination: #6D00B0 Stats: improper block termination: #6D00C0 Stats: improper block termination: #6D00E0 Stats: improper block termination: #6D0160 ** Script Error: end! word has no context ** Near: end! 377 unset! 3454 error! Actually, it's not a failure. These unterminated blocks are normal and are used in the memory manager. They are not executed, so it's never a problem. We need to remove this message so as not to confuse things. The script error is from using PRINT. The function returns a block that should not be reduced, so best to use PROBE instead. -Carl

 [17/19] from: Christophe:Coussement:mil:be at: 16-Jul-2003 12:46


[Carl said:] The file memory will not be released until the local variable is set to a different value (by a second call to foo, which will then hold the file memory until the next call, etc.) To change that, unreference the file series when done: foo: has [file] [ file: read %large-file.txt print length? file file: none ] Should this reach the same effect or better ? any clue ? foo: has [file] [ file: read %large-file.txt print length? file unset 'file ] ==xtof

 [18/19] from: g:santilli:tiscalinet:it at: 16-Jul-2003 13:21


Hi Coussement, On Wednesday, July 16, 2003, 12:46:25 PM, you wrote: CC> Should this reach the same effect or better ? any clue ? CC> foo: has [file] [ CC> file: read %large-file.txt CC> print length? file CC> unset 'file CC> ] It's the same. Even: file: 1 would be fine, as long as the "big" value you want to be GCed is no more referenced. Regards, Gabriele. -- Gabriele Santilli <[g--santilli--tiscalinet--it]> -- REBOL Programmer Amigan -- AGI L'Aquila -- REB: http://web.tiscali.it/rebol/index.r

 [19/19] from: Christophe:Coussement:mil:be at: 16-Jul-2003 14:08


Gabriele: thanks !

Notes
  • Quoted lines have been omitted from some messages.
    View the message alone to see the lines that have been omitted