FTP memory leak?
[1/13] from: amicom:sonic at: 13-Apr-2004 7:35
Hello, all. I was wondering if anyone had seen a memory leak related to FTP. I have written a backup application using Rebol. I can back up 50,000 local files without any problems, but if I try to back up ~250 files from an FTP location to a local drive, Rebol's memory usage goes through the roof and then I get a crash due to not enough memory. IIRC, someone wrote an improved FTP. Does it address this? Where can I get it? Thanks! Bohdan "Bo" Lechnowsky Lechnowsky Technical Consulting
[2/13] from: warp:reboot:ch at: 13-Apr-2004 19:22
Hi Bo, dunno about the memory leak but here you can find Romano's latest ftp handler which I'm using successfully: http://www.rebol.net/cgi-bin/projects/track.r?id=149 Will On 13 apr 2004, at 16:35, Bohdan or Rosemary Lechnowsky wrote:
[3/13] from: amicom:sonic at: 16-Apr-2004 21:07
An update: Before using Romano's patch, I was consistently only able to copy ~250 files from FTP to a local drive. After Romano's patch, I was consistently able to copy ~1500 files. And it isn't failing because of a lack of memory any longer. It is getting the "Smooth Copy has performed an illegal operation" error message or "Crash -- should not happen" from Rebol. -Bo At 07:22 PM 4/13/04 +0200, you wrote:
[4/13] from: rotenca::telvia::it at: 17-Apr-2004 11:26
Smooth Copy has performed an illegal operation I did never see this error, what is Smooth Copy? --- Ciao Romano
[5/13] from: amicom:sonic at: 17-Apr-2004 7:51
Smooth Copy is a backup application that I am writing. I have it encapped as smooth-copy.exe. It is a standard Windows error when a program crashes. -Bo At 11:26 AM 4/17/04 +0200, you wrote:
[6/13] from: rotenca:telvia:it at: 18-Apr-2004 11:30
> Smooth Copy is a backup application that I am writing. I have it encapped > as smooth-copy.exe. It is a standard Windows error when a program crashes. >
Can't you isolate the problem? Do you see memory use increasing more and more with task manager? --- Ciao Romano
[7/13] from: antonr:lexicon at: 18-Apr-2004 23:54
You should rename it to Rough Copy until this bug is gone :) Anton.
[8/13] from: amicom:sonic at: 18-Apr-2004 7:23
Anton, Maybe more appropriately, I should rename it "Smooth Copy unless it's FTP in which case it's Rough Copy" as it works perfectly well when copying files locally (source and destination). -Bo At 11:54 PM 4/18/04 +1000, you wrote:
[9/13] from: amicom:sonic at: 18-Apr-2004 7:35
Romano, I intend to try to isolate the problem. However, I have watched memory usage as it is copying several hundred files and it appears to remain relatively constant (slow increase until garbage collection kicks in when the memory usage goes back down). Somewhere before it gets near 1500 files something goes haywire with FTP. Hard to find problems when they happen infrequently during 1500 iterations, especially in a recursive application. I'll add more logging to the script along with how many levels deep in recursion it is, what the memory usage is (system/stats), and what files it is copying to see if I can isolate the problem so I can then recreate and debug it. -Bo At 11:30 AM 4/18/04 +0200, you wrote:
[10/13] from: antonr:lexicon at: 19-Apr-2004 1:36
Just out of interest, are you using read/lines anywhere? Anton.
[11/13] from: amicom:sonic at: 18-Apr-2004 9:20
Anton, No, I mostly use read-io and straight 'read for general reading (like the configuration file for example). I use open/binary/direct/read to open the source file. -Bo At 01:36 AM 4/19/04 +1000, you wrote:
[12/13] from: rotenca:telvia:it at: 18-Apr-2004 19:53
> I'll add more logging to the script along with how many levels deep in > recursion it is, what the memory usage is (system/stats), and what files it > is copying to see if I can isolate the problem so I can then recreate and > debug it.
Well, i do not know if you use it, but use my version of the rebol delete command (in 1.3 rebol stuff) . The actual version of delete has a bug with FTP. --- Ciao Romano
[13/13] from: amicom::sonic::net at: 19-Apr-2004 21:15
Just an update for those interested. Regarding the memory leak in FTP, over the weekend I was able to copy ~6500 files over FTP using Smooth Copy before the memory leak caused the program to fail. That's a definite improvement over the ~1500 files it was failing at last week. Bohdan "Bo" Lechnowsky Lechnowsky Technical Consulting