Memory usage
[1/14] from: maillist::peter::home::se at: 13-Oct-2005 23:27
Hello!
I have noticed that if I execute a script repeatedly
I will eventually ran out of memory. I have seen this
for REBOL/Core on both Linux and Windows and with
different versions.
I am almost certain that it has to do with my script
and the way REBOL allocate memory for variables. What
should I do in my script to clean up before exit?
Best regards,
Peter Carlsson
[2/14] from: SunandaDH:aol at: 14-Oct-2005 4:20
Peter:
> What should I do in my script to clean up before exit?
A couple of suggestions....
* Use recycle at various points -- it can reclaim memory from dead variables
* I tend to break applications up into small "modules" (not in the expected
REBOL sense of the word) that can be unset after use, eg:
do %init-module.r ;; loads an object called init-module that has many
functions
init-module/do-it ;; run the initialisation -- it'll set up some
global data objects
unset 'init-module ;; delete all the initialisation stuff
Sunanda.
[3/14] from: volker::nitsch::gmail::com at: 14-Oct-2005 14:17
On 10/13/05, Peter Carlsson <[maillist--peter--home--se]> wrote:
> Hello!
> I have noticed that if I execute a script repeatedly
<<quoted lines omitted: 4>>
> and the way REBOL allocate memory for variables. What
> should I do in my script to clean up before exit?
Do you use faces with rate?
In that case the timers stay active even when the window is closed,
and the window and all its contents can not be deleted.
One window more each run.
you can stop them by
hide window
usually.
> Best regards,
> Peter Carlsson
> --
> To unsubscribe from the list, just send an email to
> lists at rebol.com with unsubscribe as the subject.
>
--
-Volker
Any problem in computer science can be solved with another layer of
indirection. But that usually will create another problem.
David
Wheeler
[4/14] from: maillist:peter:home:se at: 14-Oct-2005 14:25
Volker Nitsch wrote:
Hello!
> Do you use faces with rate?
> In that case the timers stay active even when the window is closed,
<<quoted lines omitted: 3>>
> hide window
> usually.
No, I almost always use REBOL/Core.
Best regards,
Peter Carlsson
[5/14] from: greggirwin::mindspring::com at: 14-Oct-2005 9:12
Hi Peter,
PC> I have noticed that if I execute a script repeatedly
PC> I will eventually ran out of memory. I have seen this
PC> for REBOL/Core on both Linux and Windows and with
PC> different versions.
PC> I am almost certain that it has to do with my script
PC> and the way REBOL allocate memory for variables. What
PC> should I do in my script to clean up before exit?
Can you provide more information about what your script does? Hard to
say otherwise.
-- Gregg
[6/14] from: maillist:peter:home:se at: 18-Oct-2005 21:18
Gregg Irwin wrote:
> Hi Peter,
> PC> I have noticed that if I execute a script repeatedly
<<quoted lines omitted: 6>>
> Can you provide more information about what your script does? Hard to
> say otherwise.
Well, it actually is a big program which creates a few
but big lists of text strings. These lists are initially
cleared with 'a-list: copy []'. Is there a better way?
I thought that this would clear any old values and free
that memory.
Maybe I should do like Sunanda suggested in a previous
email.
Could I use 'context' for this purpose?
Best regards,
Peter Carlsson
[7/14] from: greggirwin:mindspring at: 18-Oct-2005 14:57
Hi Peter,
You can use COPY or CLEAR. If your app is running in a tight loop, you
will definitely want to give the garbage collector a chance to run. If
it doesn't have to be interactive, you can even add some WAITs in
there if you want.
RECYCLE is your friend; CONTEXT won't solve your problem.
-- Gregg
[8/14] from: lmecir:mbox:vol:cz at: 19-Oct-2005 8:42
Peter Carlsson napsal(a):
>Gregg Irwin wrote:
>>Hi Peter,
<<quoted lines omitted: 22>>
>Best regards,
>Peter Carlsson
Hi Peter,
actually I am sure, that there is nothing wrong with (a-list: copy []).
It should be sufficient to free previously allocated space for A-LIST.
RECYCLE is not meant to be used regularly by users, because the Garbage
Collector runs automatically. The best what you can do is to create a
simplified version of your script running out of memory, which might
reveal a bug either in your code or in the interpreter.
-L
[9/14] from: Christophe:Coussement:mil:be at: 19-Oct-2005 9:41
Hi list,
I did also have a lot of trouble with REBOL garbage collecting strategy.
Assuming 'recycle would work like a C deallocation function, I used it a
lot into some of my resources-consuming scripts. But one day I stated
the following:
>> stats
== 4183470
>> my-string: make string! 10000000
== ""
>> stats
== 13186209 ;-> + 10MB
>> recycle
>> stats
== 13186065 ;-> Gee! Didn't work at all ???
After a few try-and-errors, I found that first using a 'make on the word
previously to the call to 'recycle, solves the problem:
(...continuing previous code snippet)
>> my-string: make none! none
== none
>> stats
== 13186577 ;-> just checking: nothing happened
>> recycle
>> stats
== 3185714 ;-> memory obviously freed
You do not have to use a 'make none! To achieve this: a call to 'make
string! "" or whatever does the job too.
I solve most of my memory problem using this 'free-memory function:
free-memory: func ['w][set :w make none! none recycle]
You can find an interesting thread about this subject on the French
REBOL forum: http://www.codeur.org/forum/message.php?ID_Sujet=2500
Now, I do not know if this is the regular use of 'recycle, as meant by
RT. Or if this is a buggy thing... Perhaps have the list's gurus some
thoughts about that?
Anyway, I hope this will help :-)
==christophe
[10/14] from: lmecir:mbox:vol:cz at: 19-Oct-2005 10:12
Hi Christophe,
....
>I did also have a lot of trouble with REBOL garbage collecting strategy.
>Assuming 'recycle would work like a C deallocation function, I used it a
<<quoted lines omitted: 17>>
>>>
>13186065 ;-> Gee! Didn't work at all ???
This is a special case not handled by the Garbage Collector as you
correctly found out. The GC "thinks", that you are going to need this
allocated memory when you allocated it intentionally. Therefore, it will
not "shrink" the space.
>After a few try-and-errors, I found that first using a 'make on the word
>previously to the call to 'recycle, solves the problem:
>
....
>>>my-string: make none! none
>>>
>>>
>none
>
my-string: none
or
my-string: copy ""
or
unset 'my-string
would suffice.
>>>stats
>>>
<<quoted lines omitted: 5>>
>>>
>3185714 ;-> memory obviously freed
Right. RECYCLE call was not totally necessary, because it would happen
automatically sooner or later anyway.
>You do not have to use a 'make none! To achieve this: a call to 'make
>string! "" or whatever does the job too.
>
Yes.
>I solve most of my memory problem using this 'free-memory function:
>
>free-memory: func ['w][set :w make none! none recycle]
>
The shortest way may be to just write:
unset 'w
or
w: none
or
w: copy ""
or anything else not reserving too much memory as you wrote above
instead of:
free-memory w
The Garbage Collector is being called regularly anyway. If you call
RECYCLE too often, you are unnecessarily slowing down the interpreter.
>You can find an interesting thread about this subject on the French
>REBOL forum: http://www.codeur.org/forum/message.php?ID_Sujet=2500
>
I tried the URL and I got a C++ related page?
>Now, I do not know if this is the regular use of 'recycle, as meant by
>RT. Or if this is a buggy thing... Perhaps have the list's gurus some
>thoughts about that?
>
>Anyway, I hope this will help :-)
>
>Dchristophe
>
Yes, I think that those comments are helpful for somebody trying to
figure what is and what isn't collected by the GC. (Maybe we should put
it to a REBOL doc somewhere?)
-L
[11/14] from: Christophe:Coussement:mil:be at: 19-Oct-2005 10:46
Hi Ladislav,
Thank you for your precisions, as accurate as usual :-)
> >
> This is a special case not handled by the Garbage Collector as you
> correctly found out. The GC "thinks", that you are going to need this
> allocated memory when you allocated it intentionally. Therefore, it
will
> not "shrink" the space.
[[CC]] OK, I understand now
> >After a few try-and-errors, I found that first using a 'make on the
word
> >previously to the call to 'recycle, solves the problem:
> >
<<quoted lines omitted: 5>>
> unset 'my-string
> would suffice.
[[CC]] Actually, I did try those options, but without result, like
illustrated (using /view 1.3):
>> stats
== 4183470
>> my-string: make string! 10000000
== ""
>> stats
== 13186209
>> unset 'my-string
>> recycle stats
== 13186113 ;-> not freed
>> my-string: none
== none
>> recycle stats
== 13186145 ;-> not freed
>> my-string: copy ""
== ""
>> recycle stats
== 13186161 ;-> not freed
>> my-string: make none! none
== none
>> recycle stats
== 3185810 ;-> freed
But I did perhaps misunderstand some part of your explanation?
> >
> Right. RECYCLE call was not totally necessary, because it would happen
<<quoted lines omitted: 3>>
> The Garbage Collector is being called regularly anyway. If you call
> RECYCLE too often, you are unnecessarily slowing down the interpreter.
[[CC]] Indeed, as obviously shown here:
>> rec: does [start: now/time/precise i: 0 loop 1000 [i: i + 1 recycle]
print now/time/precise - start]
>> rec
0:00:03.525
>> no-rec: does [start: now/time/precise i: 0 loop 1000 [i: i + 1] print
now/time/precise - start]
>> no-rec
0:00
> >You can find an interesting thread about this subject on the French
> >REBOL forum: http://www.codeur.org/forum/message.php?ID_Sujet=2500
> >
> >
> I tried the URL and I got a C++ related page?
[[CC]] For some mysterious reasons (encoding perhaps?) some letters of
my post are stripped out and replaced by weird things :-( (or is it
Halloween approaching ?)
The correct link is:
http://www.codeur.org/forum/message.php?ID_Sujet=2500
ending with "ID_Sujet equals 2500".
> >
> Yes, I think that those comments are helpful for somebody trying to
> figure what is and what isn't collected by the GC. (Maybe we should
put
> it to a REBOL doc somewhere?)
[[CC]] OK, I will try to synthesize this thread into an article on the
REBOL Documentation Project :-)
==christophe
[12/14] from: Christophe:Coussement:mil:be at: 19-Oct-2005 10:54
> > The Garbage Collector is being called regularly anyway. If you call
> > RECYCLE too often, you are unnecessarily slowing down the
interpreter.
> [[CC]] Indeed, as obviously shown here:
>
> >> rec: does [start: now/time/precise i: 0 loop 1000 [i: i + 1
recycle]
> print now/time/precise - start]
> >> rec
> 0:00:03.525
> >> no-rec: does [start: now/time/precise i: 0 loop 1000 [i: i + 1]
print
> now/time/precise - start]
> >> no-rec
> 0:00
>
[>CC<] Just by curiosity, I tried to estimate the supplementary time
needed by 'recycle call:
>> no-rec: does [start: now/time/precise i: 0 loop 100000 [i: i + 1]
print now/time/precise - start]
>> no-rec
0:00:00.03
>> rec: does [start: now/time/precise i: 0 loop 100000 [i: i + 1
recycle] print now/time/precise - start]
>> rec
0:05:55.507
>> 0:05:55.507 / 0:00:00.03
== 11850 ;times more or...
>> 0:05:55.507 / 100000
== 0:00:00.00355507 ;... per recycle
[13/14] from: maillist:peter:home:se at: 19-Oct-2005 19:08
Hello!
I would like to thank everyone who have answered my
question. I will try the solutions as soon as possible.
Best regards,
Peter Carlsson
[14/14] from: lmecir:mbox:vol:cz at: 20-Oct-2005 14:47
Coussement Christophe napsal(a):
....
>But I did perhaps misunderstand some part of your explanation?
>
Actually not. The fact is, that I didn't read thoroughly enough what you
wrote. I posted your findings to RAMBO, because I think, that it
deserves to be examined more thoroughly.
Thanks
-Ladislav
Notes
- Quoted lines have been omitted from some messages.
View the message alone to see the lines that have been omitted