A collection of garbage
[1/19] from: patrick::philipot::laposte::net at: 8-Jul-2003 13:46
Hi List
I often code functions returning a block.
What happens to all these blocks?
Are they "garbage collected"? I doubt it ...
Imagine a code like this one:
gimme: func[n][b: copy [] for i 1 n 1 [append b i] b]
print gimme 5
print gimme 15
print gimme 10
As I see it, there are 3 blocks that will stay in memory forever.
And this slightly different version seems not better:
gimme: func[n /local b][b: copy [] for i 1 n 1 [append b i] b]
The best I can think of is to provide a block to the function:
gimme: func[n b [block!]][clear b for i 1 n 1 [append b i] b]
result: copy []
print gimme 5 result
print gimme 15 result
print gimme 10 result
Any inspired comments?
Regards
Patrick
Regards
Patrick
[2/19] from: g:santilli:tiscalinet:it at: 8-Jul-2003 14:44
Hi Patrick,
On Tuesday, July 8, 2003, 1:46:17 PM, you wrote:
p> What happens to all these blocks?
p> Are they "garbage collected"? I doubt it ...
They will, if they are no more referenced.
Regards,
Gabriele.
--
Gabriele Santilli <[g--santilli--tiscalinet--it]> -- REBOL Programmer
Amigan -- AGI L'Aquila -- REB: http://web.tiscali.it/rebol/index.r
[3/19] from: antonr:iinet:au at: 9-Jul-2003 0:29
Well, we could probably check whether unreferenced
blocks are recycled by using the new 'stats function:
>> ? stats
USAGE:
STATS /pools /types /series /frames /recycle
DESCRIPTION:
System statistics. Default is to return total memory used.
STATS is a native value.
REFINEMENTS:
/pools -- Returns: width units free-units units-per-alloc segments
mem-in-use
/types -- Returns: Datatype count
/series -- Returns: total blocks strings other avail free expansions
/frames -- Returns: total used unused free values-total
/recycle -- Returns: count series-total series-last frames-total
frames-last ballast
Anton.
[4/19] from: patrick:philipot:laposte at: 8-Jul-2003 17:09
Hi Gabrielle,
As Fox Mulder would say "I want to believe".
But it's a very short answer to a very big problem ! Has anyone more on this
subject?
Regards
Patrick
[5/19] from: maximo:meteorstudios at: 8-Jul-2003 13:27
excuse me for butting in ;-)
Iconify a view app, and you will see that its memory consumption will drop !
you can see this with the system allocation tool.
I am not sure if its rebol that's de-allocating its ram or if its windows that hidding
that app's deep in cache files... In a typical test My app goes from 30MB to ~4-5MB and
then back to 30MB when its de-iconified...
Just adding a little mystery to the MEM issue... ;-)
has anyone tested if any memory is freed when a call to clear is done ?
-max
-----------
meteor Studios, T.D.
-----------
Strong enough for a man, but made for a woman
[6/19] from: rotenca:telvia:it at: 8-Jul-2003 19:12
> But it's a very short answer to a very big problem ! Has anyone more on this
> subject?
1) I am not sure that Rebol releases allocated memory to the system(i think it
never happens).
Use system/stats to check ebol memory allocation.
2) I am sure that Rebol re-use its no more referenced memory.
You can make some tests about this with
http://www.reboltech.com/library/scripts/mem2.r
---
Ciao
Romano
[7/19] from: maximo:meteorstudios at: 8-Jul-2003 14:38
I read my mail and I just wanted to precise that the "system allocation tool", really
refers to "Windows Task Manager"... sorry for the bad wording...
[8/19] from: rotenca:telvia:it at: 8-Jul-2003 20:40
> 1) I am not sure that Rebol releases allocated memory to the system(i think
it
> never happens).
I was wrong. It releases some memory under some conditions. But with an hard
to predict logic.
---
Ciao
Romano
[9/19] from: nitsch-lists:netcologne at: 8-Jul-2003 21:21
Am Dienstag, 8. Juli 2003 19:27 schrieb Maxim Olivier-Adlhoch:
> excuse me for butting in ;-)
>
> Iconify a view app, and you will see that its memory consumption will drop
> ! you can see this with the system allocation tool.
>
> I am not sure if its rebol that's de-allocating its ram or if its windows
> that hidding that app's deep in cache files... In a typical test My app
> goes from 30MB to ~4-5MB and then back to 30MB when its de-iconified...
>
You could run a test with /view and /core.
I expect they comparable in size when /view is iconified.
It holds a lot of 24-bit double-buffers AFAIK
> Just adding a little mystery to the MEM issue... ;-)
>
-Volker
[10/19] from: nitsch-lists:netcologne at: 8-Jul-2003 21:19
Am Dienstag, 8. Juli 2003 17:09 schrieb patrick:
> Hi Gabrielle,
>
> As Fox Mulder would say "I want to believe".
>
> But it's a very short answer to a very big problem ! Has anyone more on
> this subject?
>
If its a very big problem, Gabriele would have noticed it and published
workarounds. So its collected :)
rebol waits until there is some stuff to collect, usually some MB.
Then it collects a lot of stuff at once.
Like people do with garbage ;)
> Regards
> Patrick
<<quoted lines omitted: 18>>
> > Amigan -- AGI L'Aquila -- REB: http://web.tiscali.it/rebol/index.r
> >
-Volker
[11/19] from: carl:rebol at: 8-Jul-2003 20:06
The logic is fairly complicated, but it's also quite fast.
For example, you might be using 50MB of memory, but not
even notice when the GC happens.
There are many REBOL programs that run for months (years?)
at a time without restarting, so we know the GC is stable.
(There are a few exceptions, in operations like TRY on OPEN
failures, etc.)
If you're curious about your code's memory, just put your
code into a loop and print memory stats every so often.
(Use a newer version of REBOL for better accuracy on stats.)
loop 10000 [
your code
print ["Mem:" stats]
]
You'll see the mem usage top out after a while (unless you
are doing things like continuously appending to series, such
as when keeping a log).
The two basic memory methods in REBOL are:
* Small allocations (values, blocks, etc.) are managed by
REBOL for speed reasons. They are efficiently recycled.
* Large allocations (files, images, sounds, big blocks, etc.)
are allocated thru the native OS, because we figure it knows
better the characteristics of its VM paging model and can
optimize the allocation.
So, here's what programmers need to remember:
* If you're just using normal REBOL values and small series,
just sit back and relax.
* If you allocate large memory objects (strings, images,
blocks, etc.), you may want to un-reference them when you
are done with them. For example:
file: read %large-file.txt
do some-stuff-on-file
file: none ; allow file buffer to GC eventually
This is not required, but it will help reduce memory usage.
IMPORTANT: in the example above, if the file variable is a
local variable to a function, DO NOT ASSUME that it will be
un-referenced automatically. By default, the first frame
(first level of call) of local variables for a function remain
referenced. For example, if you have a function:
foo: has [file] [
file: read %large-file.txt
print length? file
]
The file memory will not be released until the local variable
is set to a different value (by a second call to foo, which
will then hold the file memory until the next call, etc.)
To change that, unreference the file series when done:
foo: has [file] [
file: read %large-file.txt
print length? file
file: none
]
-Carl
[12/19] from: fsievert:uos at: 9-Jul-2003 10:22
Hi Carl,
there is still a strange thing with nested foreach, do/next and reduce's.
test: func [:looper x] [
if x < 100 [looper z [1] [
test looper x + 1
prin "."
]]]
[test repeat 1] and [test foreach 1] are doing exactly the same thing, but
loop 22 [test repeat 1]
works, while
loop 22 [test foreach 1]
crashes.
Turn recycler off to see both working. You can see the same problem
with do/next and reduce.
Frank
On Tue, 8 Jul 2003, Carl at REBOL wrote:
[13/19] from: cyphre:seznam:cz at: 10-Jul-2003 9:04
Hi Carl, Frank and all,
> ----- Original Message -----
> From: "Frank Sievertsen" <[fsievert--uos--de]>
<<quoted lines omitted: 6>>
> Frank
> ......
What about this? Just try it to paste into console with and without
commented "recycle/off" line:
-----------------begginning of code-------------------
address?: function [
{get the address of a string}
s [any-string!]
] [address] [
s: make struct! [s [string!]] reduce [s]
address: make struct! [i [integer!]] none
change third address third s
address/i
]
get-mem?: function [
{get the byte from a memory address}
address [integer!]
/nts {a null-terminated string}
/part {a binary with a specified length}
length [integer!]
] [m] [
address: make struct! [i [integer!]] reduce [address]
if nts [
m: make struct! [s [string!]] none
change third m third address
return m/s
]
if part [
m: head insert/dup copy [] [. [char!]] length
m: make struct! compose/deep [bin [struct! (reduce [m])]] none
change third m third address
return to string! third m/bin
]
m: make struct! [c [struct! [chr [char!]]]] none
change third m third address
m/c/chr
]
img: make image! 512x512
;recycle/off ;Warning! should allocate about 200MB of memory but don't crash
loop 4 [
get-mem?/part address? img length? img
]
----------------------------------end of
code ---------------------------------
Looks like there are still problems with the GC??
regards,
Cyphre
[14/19] from: lmecir:mbox:vol:cz at: 10-Jul-2003 12:48
Hi Cyphre and Gabriele,
I changed the GET-MEM? function implementation slightly to use a "truly dynamic" local
variable.
I found out, that the crash occurs even if we don't use the cycle:
address?: function [
{get the address of a string}
s [any-string!]
] [address] [
s: make struct! [s [string!]] reduce [s]
address: make struct! [i [integer!]] none
change third address third s
address/i
]
get-mem?: func [
{get the byte from a memory address}
address [integer!]
/nts {a null-terminated string}
/part {a binary with a specified length}
length [integer!]
] [
use [m] [
address: make struct! [i [integer!]] reduce [address]
if nts [
m: make struct! [s [string!]] none
change third m third address
return m/s
]
if part [
m: head insert/dup copy [] [. [char!]] length
m: make struct! compose/deep [bin [struct! (reduce [m])]] none
change third m third address
return first reduce [third m/bin m: none]
]
m: make struct! [c [struct! [chr [char!]]]] none
change third m third address
m/c/chr
]
]
img: make image! 512x512
recycle/off ;Warning! should allocate about xxxMB of memory
get-mem?/part probe address? img length? img
get-mem?/part probe address? img length? img
recycle
get-mem?/part probe address? img length? img
-L
[15/19] from: patrick:philipot:laposte at: 12-Jul-2003 11:15
Hi List
Stats/types fails when used just after starting Rebol.
REBOL/View 1.2.10.3.1 30-May-2003
Copyright 2000-2002 REBOL Technologies. All rights reserved.
...
License ID: 1-01102-240
>> print stats/types
Stats: improper block termination: #6D00A0
Stats: improper block termination: #6D00B0
Stats: improper block termination: #6D00C0
Stats: improper block termination: #6D00E0
Stats: improper block termination: #6D0160
** Script Error: end! word has no context
** Near: end! 377 unset! 3454 error!
>>
Regards
Patrick
[16/19] from: Carl:rebol at: 12-Jul-2003 2:43
Hi Patrick,
About:
>> print stats/types Stats: improper block termination: #6D00A0
Stats: improper block termination: #6D00B0
Stats: improper block termination: #6D00C0
Stats: improper block termination: #6D00E0
Stats: improper block termination: #6D0160
** Script Error: end! word has no context
** Near: end! 377 unset! 3454 error!
Actually, it's not a failure. These unterminated blocks are normal and are used in
the memory manager. They are not executed, so it's never a problem. We need to
remove this message so as not to confuse things.
The script error is from using PRINT. The function returns a block that should
not be reduced, so best to use PROBE instead.
-Carl
[17/19] from: Christophe:Coussement:mil:be at: 16-Jul-2003 12:46
[Carl said:]
The file memory will not be released until the local variable
is set to a different value (by a second call to foo, which
will then hold the file memory until the next call, etc.)
To change that, unreference the file series when done:
foo: has [file] [
file: read %large-file.txt
print length? file
file: none
]
Should this reach the same effect or better ? any clue ?
foo: has [file] [
file: read %large-file.txt
print length? file
unset 'file
]
==xtof
[18/19] from: g:santilli:tiscalinet:it at: 16-Jul-2003 13:21
Hi Coussement,
On Wednesday, July 16, 2003, 12:46:25 PM, you wrote:
CC> Should this reach the same effect or better ? any clue ?
CC> foo: has [file] [
CC> file: read %large-file.txt
CC> print length? file
CC> unset 'file
CC> ]
It's the same. Even:
file: 1
would be fine, as long as the "big" value you want to be GCed is
no more referenced.
Regards,
Gabriele.
--
Gabriele Santilli <[g--santilli--tiscalinet--it]> -- REBOL Programmer
Amigan -- AGI L'Aquila -- REB: http://web.tiscali.it/rebol/index.r
[19/19] from: Christophe:Coussement:mil:be at: 16-Jul-2003 14:08
Gabriele: thanks !
Notes
- Quoted lines have been omitted from some messages.
View the message alone to see the lines that have been omitted