FTP memory leak?
[1/13] from: amicom:sonic at: 13-Apr-2004 7:35
Hello, all.
I was wondering if anyone had seen a memory leak related to FTP. I have
written a backup application using Rebol. I can back up 50,000 local files
without any problems, but if I try to back up ~250 files from an FTP
location to a local drive, Rebol's memory usage goes through the roof and
then I get a crash due to not enough memory.
IIRC, someone wrote an improved FTP. Does it address this? Where can I
get it?
Thanks!
Bohdan "Bo" Lechnowsky
Lechnowsky Technical Consulting
[2/13] from: warp:reboot:ch at: 13-Apr-2004 19:22
Hi Bo,
dunno about the memory leak but here you can find Romano's latest
ftp handler which I'm using successfully:
http://www.rebol.net/cgi-bin/projects/track.r?id=149
Will
On 13 apr 2004, at 16:35, Bohdan or Rosemary Lechnowsky wrote:
[3/13] from: amicom:sonic at: 16-Apr-2004 21:07
An update:
Before using Romano's patch, I was consistently only able to copy ~250
files from FTP to a local drive. After Romano's patch, I was consistently
able to copy ~1500 files. And it isn't failing because of a lack of memory
any longer. It is getting the "Smooth Copy has performed an illegal
operation" error message or "Crash -- should not happen" from Rebol.
-Bo
At 07:22 PM 4/13/04 +0200, you wrote:
[4/13] from: rotenca::telvia::it at: 17-Apr-2004 11:26
Smooth Copy has performed an illegal operation
I did never see this error, what is Smooth Copy?
---
Ciao
Romano
[5/13] from: amicom:sonic at: 17-Apr-2004 7:51
Smooth Copy is a backup application that I am writing. I have it encapped
as smooth-copy.exe. It is a standard Windows error when a program crashes.
-Bo
At 11:26 AM 4/17/04 +0200, you wrote:
[6/13] from: rotenca:telvia:it at: 18-Apr-2004 11:30
Hi,
> Smooth Copy is a backup application that I am writing. I have it encapped
> as smooth-copy.exe. It is a standard Windows error when a program crashes.
>
Can't you isolate the problem?
Do you see memory use increasing more and more with task manager?
---
Ciao
Romano
[7/13] from: antonr:lexicon at: 18-Apr-2004 23:54
You should rename it to Rough Copy until this bug
is gone :)
Anton.
[8/13] from: amicom:sonic at: 18-Apr-2004 7:23
Anton,
Maybe more appropriately, I should rename it "Smooth Copy unless it's FTP
in which case it's Rough Copy" as it works perfectly well when copying
files locally (source and destination).
-Bo
At 11:54 PM 4/18/04 +1000, you wrote:
[9/13] from: amicom:sonic at: 18-Apr-2004 7:35
Romano,
I intend to try to isolate the problem. However, I have watched memory
usage as it is copying several hundred files and it appears to remain
relatively constant (slow increase until garbage collection kicks in when
the memory usage goes back down). Somewhere before it gets near 1500 files
something goes haywire with FTP. Hard to find problems when they happen
infrequently during 1500 iterations, especially in a recursive application.
I'll add more logging to the script along with how many levels deep in
recursion it is, what the memory usage is (system/stats), and what files it
is copying to see if I can isolate the problem so I can then recreate and
debug it.
-Bo
At 11:30 AM 4/18/04 +0200, you wrote:
[10/13] from: antonr:lexicon at: 19-Apr-2004 1:36
Just out of interest, are you using read/lines anywhere?
Anton.
[11/13] from: amicom:sonic at: 18-Apr-2004 9:20
Anton,
No, I mostly use read-io and straight 'read for general reading (like the
configuration file for example). I use open/binary/direct/read to open the
source file.
-Bo
At 01:36 AM 4/19/04 +1000, you wrote:
[12/13] from: rotenca:telvia:it at: 18-Apr-2004 19:53
Hi, Bo
> I'll add more logging to the script along with how many levels deep in
> recursion it is, what the memory usage is (system/stats), and what files it
> is copying to see if I can isolate the problem so I can then recreate and
> debug it.
Well, i do not know if you use it, but use my version of the rebol delete
command (in 1.3 rebol stuff) . The actual version of delete has a bug with
FTP.
---
Ciao
Romano
[13/13] from: amicom::sonic::net at: 19-Apr-2004 21:15
FTP problem
Just an update for those interested.
Regarding the memory leak in FTP, over the weekend I was able to copy ~6500
files over FTP using Smooth Copy before the memory leak caused the program
to fail. That's a definite improvement over the ~1500 files it was failing
at last week.
Bohdan "Bo" Lechnowsky
Lechnowsky Technical Consulting