Mailing List Archive: 49091 messages
  • Home
  • Script library
  • AltME Archive
  • Mailing list
  • Articles Index
  • Site search
 

Memory usage

 [1/14] from: maillist::peter::home::se at: 13-Oct-2005 23:27


Hello! I have noticed that if I execute a script repeatedly I will eventually ran out of memory. I have seen this for REBOL/Core on both Linux and Windows and with different versions. I am almost certain that it has to do with my script and the way REBOL allocate memory for variables. What should I do in my script to clean up before exit? Best regards, Peter Carlsson

 [2/14] from: SunandaDH:aol at: 14-Oct-2005 4:20


Peter:
> What should I do in my script to clean up before exit?
A couple of suggestions.... * Use recycle at various points -- it can reclaim memory from dead variables * I tend to break applications up into small "modules" (not in the expected REBOL sense of the word) that can be unset after use, eg: do %init-module.r ;; loads an object called init-module that has many functions init-module/do-it ;; run the initialisation -- it'll set up some global data objects unset 'init-module ;; delete all the initialisation stuff Sunanda.

 [3/14] from: volker::nitsch::gmail::com at: 14-Oct-2005 14:17


On 10/13/05, Peter Carlsson <[maillist--peter--home--se]> wrote:
> Hello! > I have noticed that if I execute a script repeatedly
<<quoted lines omitted: 4>>
> and the way REBOL allocate memory for variables. What > should I do in my script to clean up before exit?
Do you use faces with rate? In that case the timers stay active even when the window is closed, and the window and all its contents can not be deleted. One window more each run. you can stop them by hide window usually.
> Best regards, > Peter Carlsson > -- > To unsubscribe from the list, just send an email to > lists at rebol.com with unsubscribe as the subject. >
-- -Volker Any problem in computer science can be solved with another layer of indirection. But that usually will create another problem. David Wheeler

 [4/14] from: maillist:peter:home:se at: 14-Oct-2005 14:25


Volker Nitsch wrote: Hello!
> Do you use faces with rate? > In that case the timers stay active even when the window is closed,
<<quoted lines omitted: 3>>
> hide window > usually.
No, I almost always use REBOL/Core. Best regards, Peter Carlsson

 [5/14] from: greggirwin::mindspring::com at: 14-Oct-2005 9:12


Hi Peter, PC> I have noticed that if I execute a script repeatedly PC> I will eventually ran out of memory. I have seen this PC> for REBOL/Core on both Linux and Windows and with PC> different versions. PC> I am almost certain that it has to do with my script PC> and the way REBOL allocate memory for variables. What PC> should I do in my script to clean up before exit? Can you provide more information about what your script does? Hard to say otherwise. -- Gregg

 [6/14] from: maillist:peter:home:se at: 18-Oct-2005 21:18


Gregg Irwin wrote:
> Hi Peter, > PC> I have noticed that if I execute a script repeatedly
<<quoted lines omitted: 6>>
> Can you provide more information about what your script does? Hard to > say otherwise.
Well, it actually is a big program which creates a few but big lists of text strings. These lists are initially cleared with 'a-list: copy []'. Is there a better way? I thought that this would clear any old values and free that memory. Maybe I should do like Sunanda suggested in a previous email. Could I use 'context' for this purpose? Best regards, Peter Carlsson

 [7/14] from: greggirwin:mindspring at: 18-Oct-2005 14:57


Hi Peter, You can use COPY or CLEAR. If your app is running in a tight loop, you will definitely want to give the garbage collector a chance to run. If it doesn't have to be interactive, you can even add some WAITs in there if you want. RECYCLE is your friend; CONTEXT won't solve your problem. -- Gregg

 [8/14] from: lmecir:mbox:vol:cz at: 19-Oct-2005 8:42


Peter Carlsson napsal(a):
>Gregg Irwin wrote: >>Hi Peter,
<<quoted lines omitted: 22>>
>Best regards, >Peter Carlsson
Hi Peter, actually I am sure, that there is nothing wrong with (a-list: copy []). It should be sufficient to free previously allocated space for A-LIST. RECYCLE is not meant to be used regularly by users, because the Garbage Collector runs automatically. The best what you can do is to create a simplified version of your script running out of memory, which might reveal a bug either in your code or in the interpreter. -L

 [9/14] from: Christophe:Coussement:mil:be at: 19-Oct-2005 9:41


Hi list, I did also have a lot of trouble with REBOL garbage collecting strategy. Assuming 'recycle would work like a C deallocation function, I used it a lot into some of my resources-consuming scripts. But one day I stated the following:
>> stats
== 4183470
>> my-string: make string! 10000000
== ""
>> stats
== 13186209 ;-> + 10MB
>> recycle >> stats
== 13186065 ;-> Gee! Didn't work at all ??? After a few try-and-errors, I found that first using a 'make on the word previously to the call to 'recycle, solves the problem: (...continuing previous code snippet)
>> my-string: make none! none
== none
>> stats
== 13186577 ;-> just checking: nothing happened
>> recycle >> stats
== 3185714 ;-> memory obviously freed You do not have to use a 'make none! To achieve this: a call to 'make string! "" or whatever does the job too. I solve most of my memory problem using this 'free-memory function: free-memory: func ['w][set :w make none! none recycle] You can find an interesting thread about this subject on the French REBOL forum: http://www.codeur.org/forum/message.php?ID_Sujet=2500 Now, I do not know if this is the regular use of 'recycle, as meant by RT. Or if this is a buggy thing... Perhaps have the list's gurus some thoughts about that? Anyway, I hope this will help :-) ==christophe

 [10/14] from: lmecir:mbox:vol:cz at: 19-Oct-2005 10:12


Hi Christophe, ....
>I did also have a lot of trouble with REBOL garbage collecting strategy. >Assuming 'recycle would work like a C deallocation function, I used it a
<<quoted lines omitted: 17>>
>>> >13186065 ;-> Gee! Didn't work at all ???
This is a special case not handled by the Garbage Collector as you correctly found out. The GC "thinks", that you are going to need this allocated memory when you allocated it intentionally. Therefore, it will not "shrink" the space.
>After a few try-and-errors, I found that first using a 'make on the word >previously to the call to 'recycle, solves the problem: >
....
>>>my-string: make none! none >>> >>> >none >
my-string: none or my-string: copy "" or unset 'my-string would suffice.
>>>stats >>>
<<quoted lines omitted: 5>>
>>> >3185714 ;-> memory obviously freed
Right. RECYCLE call was not totally necessary, because it would happen automatically sooner or later anyway.
>You do not have to use a 'make none! To achieve this: a call to 'make >string! "" or whatever does the job too. >
Yes.
>I solve most of my memory problem using this 'free-memory function: > >free-memory: func ['w][set :w make none! none recycle] >
The shortest way may be to just write: unset 'w or w: none or w: copy "" or anything else not reserving too much memory as you wrote above instead of: free-memory w The Garbage Collector is being called regularly anyway. If you call RECYCLE too often, you are unnecessarily slowing down the interpreter.
>You can find an interesting thread about this subject on the French >REBOL forum: http://www.codeur.org/forum/message.php?ID_Sujet=2500 >
I tried the URL and I got a C++ related page?
>Now, I do not know if this is the regular use of 'recycle, as meant by >RT. Or if this is a buggy thing... Perhaps have the list's gurus some >thoughts about that? > >Anyway, I hope this will help :-) > >Dchristophe >
Yes, I think that those comments are helpful for somebody trying to figure what is and what isn't collected by the GC. (Maybe we should put it to a REBOL doc somewhere?) -L

 [11/14] from: Christophe:Coussement:mil:be at: 19-Oct-2005 10:46


Hi Ladislav, Thank you for your precisions, as accurate as usual :-)
> > > This is a special case not handled by the Garbage Collector as you > correctly found out. The GC "thinks", that you are going to need this > allocated memory when you allocated it intentionally. Therefore, it
will
> not "shrink" the space.
[[CC]] OK, I understand now
> >After a few try-and-errors, I found that first using a 'make on the
word
> >previously to the call to 'recycle, solves the problem: > >
<<quoted lines omitted: 5>>
> unset 'my-string > would suffice.
[[CC]] Actually, I did try those options, but without result, like illustrated (using /view 1.3):
>> stats
== 4183470
>> my-string: make string! 10000000
== ""
>> stats
== 13186209
>> unset 'my-string >> recycle stats
== 13186113 ;-> not freed
>> my-string: none
== none
>> recycle stats
== 13186145 ;-> not freed
>> my-string: copy ""
== ""
>> recycle stats
== 13186161 ;-> not freed
>> my-string: make none! none
== none
>> recycle stats
== 3185810 ;-> freed But I did perhaps misunderstand some part of your explanation?
> > > Right. RECYCLE call was not totally necessary, because it would happen
<<quoted lines omitted: 3>>
> The Garbage Collector is being called regularly anyway. If you call > RECYCLE too often, you are unnecessarily slowing down the interpreter.
[[CC]] Indeed, as obviously shown here:
>> rec: does [start: now/time/precise i: 0 loop 1000 [i: i + 1 recycle]
print now/time/precise - start]
>> rec
0:00:03.525
>> no-rec: does [start: now/time/precise i: 0 loop 1000 [i: i + 1] print
now/time/precise - start]
>> no-rec
0:00
> >You can find an interesting thread about this subject on the French > >REBOL forum: http://www.codeur.org/forum/message.php?ID_Sujet=2500 > > > > > I tried the URL and I got a C++ related page?
[[CC]] For some mysterious reasons (encoding perhaps?) some letters of my post are stripped out and replaced by weird things :-( (or is it Halloween approaching ?) The correct link is: http://www.codeur.org/forum/message.php?ID_Sujet=2500 ending with "ID_Sujet equals 2500".
> > > Yes, I think that those comments are helpful for somebody trying to > figure what is and what isn't collected by the GC. (Maybe we should
put
> it to a REBOL doc somewhere?)
[[CC]] OK, I will try to synthesize this thread into an article on the REBOL Documentation Project :-) ==christophe

 [12/14] from: Christophe:Coussement:mil:be at: 19-Oct-2005 10:54


> > The Garbage Collector is being called regularly anyway. If you call > > RECYCLE too often, you are unnecessarily slowing down the
interpreter.
> [[CC]] Indeed, as obviously shown here: > > >> rec: does [start: now/time/precise i: 0 loop 1000 [i: i + 1
recycle]
> print now/time/precise - start] > >> rec > 0:00:03.525 > >> no-rec: does [start: now/time/precise i: 0 loop 1000 [i: i + 1]
print
> now/time/precise - start] > >> no-rec > 0:00 >
[>CC<] Just by curiosity, I tried to estimate the supplementary time needed by 'recycle call:
>> no-rec: does [start: now/time/precise i: 0 loop 100000 [i: i + 1]
print now/time/precise - start]
>> no-rec
0:00:00.03
>> rec: does [start: now/time/precise i: 0 loop 100000 [i: i + 1
recycle] print now/time/precise - start]
>> rec
0:05:55.507
>> 0:05:55.507 / 0:00:00.03
== 11850 ;times more or...
>> 0:05:55.507 / 100000
== 0:00:00.00355507 ;... per recycle

 [13/14] from: maillist:peter:home:se at: 19-Oct-2005 19:08


Hello! I would like to thank everyone who have answered my question. I will try the solutions as soon as possible. Best regards, Peter Carlsson

 [14/14] from: lmecir:mbox:vol:cz at: 20-Oct-2005 14:47


Coussement Christophe napsal(a): ....
>But I did perhaps misunderstand some part of your explanation? >
Actually not. The fact is, that I didn't read thoroughly enough what you wrote. I posted your findings to RAMBO, because I think, that it deserves to be examined more thoroughly. Thanks -Ladislav

Notes
  • Quoted lines have been omitted from some messages.
    View the message alone to see the lines that have been omitted