Mailing List Archive: 49091 messages
  • Home
  • Script library
  • AltME Archive
  • Mailing list
  • Articles Index
  • Site search
 

FTP problems -- mystery.

 [1/5] from: joel::neely::fedex::com at: 25-Sep-2000 14:39


I had an HP-UX server running a REBOL script in a cron job every night to pull a data file from a wNT box. That script does foo: read ftp://userid:[password--1--2--3--4]/path/file.txt (wrapped in a try block), then post-processes the data before writing it to a local file. This had been running non-stop since mid-April. On 26 Jun 2000 I upgraded that HP-UX box from 2.2 to 2.3 (don't try to read that aloud! ;-). Everything continued to work properly. On 10 Aug 2000 the above statement began consistently failing. I can run the standard ftp utility from the command line and retrieve the file (with almost instantaneous response), but the REBOL script fails whether run manually or via cron. When I run REBOL interactively and try the same command, with tracing enabled, I get the following:
>> system/schemes/ftp/passive: on
== true (The above inspired by Carl's note below, but the same happens without that intervention.)
>> foo: read ftp://userid:[password--1--2--3--4]/path/file.txt
connecting to: 1.2.3.4 ** User Error: Server error: tcp HTTP/1.0 408 Request Time-out. ** Where: foo: read ftp://userid:[password--1--2--3--4]/path/file.txt
>> trace/net on >> foo: read ftp://userid:[password--1--2--3--4]/path/file.txt
URL Parse: userid password 1.2.3.4 none path/ file.txt Net-log: ["Opening tcp for" FTP] connecting to: 1.2.3.4 Net-log: [ none ["220" "230"]] ** User Error: Server error: tcp HTTP/1.0 408 Request Time-out. ** Where: foo: read ftp://userid:[password--1--2--3--4]/path/file.txt
>> system/schemes/ftp/passive
== true Any suggestions from the REBOLnetworkingwizards about what I should try next? -jn- [carl--rebol--com] wrote:

 [2/5] from: kracik:mbox:dkm:cz at: 25-Sep-2000 22:48


Hi, I'm not a REBOLnetworkingwizard, but from the error message tcp HTTP/1.0 408 Request Time-out. it seems that a HTTP, not FTP server listens at default FTP port 21, so the WinNT server configuration may be responsible for it. Causes of such mysterious errors are easily discovered with a packet sniffer and analyzer, where you can see the whole TCP communication. hope this helps, -- Michal Kracik [joel--neely--fedex--com] wrote:

 [3/5] from: joel:neely:fedex at: 25-Sep-2000 16:20


Thanks, Michal... But I believe the services on the host(s) are correct. It seems that I get a similar message whenever I try to connect from REBOL (either on the HP-UX box or my desktop box running Linux) to ANY ftp server. However, a transcript from a console ftp session (to the same host I'm trying to hit from REBOL) makes it appear that a "normal" ftp service is running on the host's port 21. $ ftp 1.2.3.4 Connected to 1.2.3.4. 220 HOSTNAME Microsoft FTP Service (Version 4.0). Name (1.2.3.4:user): userid 331 Password required for userid. Password: 230-Hello, welcome to HOSTNAME. 230 User userid logged in. Remote system type is Windows_NT. ftp> ls 200 PORT command successful. 150 Opening ASCII mode data connection for /bin/ls. 09-25-00 07:23AM <DIR> downloads 09-24-00 06:00PM <DIR> xxxxxxx 08-24-00 09:01AM <DIR> yyyyyyy 05-12-00 03:02PM <DIR> zzzzzzz 06-30-00 05:38PM <DIR> wwwwwww ftp> quit 221 Goodbye! $ I get a similar response when connecting to the HP-UX server from my Linux box, and I just checked /etc/services and /etc/inetd.conf to verify that ftp really is running on port 21. $ ftp my-hpux-box Connected to my-hpux-box. 220 my-hpux-box FTP server (Version 1.7.212.2 Tue Apr 21 12:14:46 GMT 1998) ready. Name (my-hpux-box:zzuserzz): xxuserxx 331 Password required for xxuserxx. Password: 230 User xxuserxx logged in. Remote system type is UNIX. Using binary mode to transfer files. ftp> ls 200 PORT command successful. 150 Opening ASCII mode data connection for /usr/bin/ls. . . long ls output suppressed here . 226 Transfer complete. ftp> get user.r junkfilekillme local: junkfilekillme remote: user.r 200 PORT command successful. 150 Opening BINARY mode data connection for user.r (152 bytes). 226 Transfer complete. 152 bytes received in 0.00718 secs (21 Kbytes/sec) ftp> quit 221 Goodbye. $ But then, from my desktop Linux box...
>> system/version
== 2.3.0.4.2 (just FYI)
>> foo: read http://www.rebol.com/
URL Parse: none none www.rebol.com none none none Net-log: ["Opening tcp for" HTTP] connecting to: www.rebol.com Net-log: {GET http://www.rebol.com/ HTTP/1.0 Accept: */* Connection: close User-Agent: REBOL 2.3.0.4.2 Host: www.rebol.com } Net-log: "HTTP/1.0 200 OK" Net-log: ["low level read of " 11210 "bytes"] == {<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN"> <HTML> <HEAD> <META HTTP-EQUIV="Content-Type" CONTENT="text/html;CHARSET=iso-... (OK. We've verified REBOL talking to the network.)
>> foo: read ftp://xxuseridxx:[xxpasswordxx--my-hpux-host]/
URL Parse: xxuseridxx xxpasswordxx my-hpux-host none none none Net-log: ["Opening tcp for" FTP] connecting to: my-hpux-host Net-log: [ none ["220" "230"]] ** User Error: Server error: tcp HTTP/1.0 408 Request Time-out. ** Where: foo: read ftp://xxuseridxx:[xxpasswordxx--my-hpux-host]/
>>
??? - [kracik--mbox--dkm--cz] wrote:

 [4/5] from: holger:rebol at: 25-Sep-2000 15:08


On Mon, Sep 25, 2000 at 04:20:10PM -0500, [joel--neely--fedex--com] wrote:
> Thanks, Michal... > > But I believe the services on the host(s) are correct. > > It seems that I get a similar message whenever I try to connect from > REBOL (either on the HP-UX box or my desktop box running Linux) > to ANY ftp server.
It looks like you have configured a generic proxy server for REBOL, and that server may no longer exist or may not be working correctly. That also explains why all FTP connections appear to be based on HTTP. Please check the set-net call in your user.r file and, if you don't need a proxy server, change "generic" to "none". -- Holger Kruse [holger--rebol--com]

 [5/5] from: joel:neely:fedex at: 25-Sep-2000 19:43


I'm passing this status report back to the list in case someone else gets bitten by a similar problem. SHORT VERSION: I have enough of a clue to get a workaround going, but still have concerns about how well REBOL and the corporate proxy server (Squid) are getting along. I also have what appears to be a bug in REBOL's handling of .../proxy/bypass entries. LONG VERSION: That thunderous CLICK! you heard a while ago was the light going on... I tracked this down with RickM, our intrepid sysadmin, looking over my shoulder (credit where credit is due). As Holger suspected, I did have a generic proxy server configured. This had been working since mid-April, and was necessary to get 'net traffic through our firewall (little useful things like 'upgrade!) The server software (Squid/2.3.STABLE2) and/or its configuration have changed in a way that is no longer compatible with REBOL. I can still hit the desired ftp server from within Netscape by using a URL of the form ftp://userid:[password--1--2--3--4]/ and get a listing of the top-level ftp directory space. If I use only the URL ftp://1.2.3.4/ I get an error message from Squid stating (correctly) that the server in question does not allow anonymous connections. Squid's error message, of course, is returned as HTML. If I disable proxy configuration for REBOL (comment out the set-net call in user.r) the ftp connection works. Re-enabling the proxy setting (de-commenting set-net and restarting REBOL) gets me the timeout again. Soooo... Turning to Carl's draft version of REBOL Network Protocols, I tried adding to user.r (after the set-net call) the line system/schemes/default/proxy/bypass: ["1.2.*" "*.fedex.com"] (where the "1.2." was actually the prefix of the class B network in which the long-sought ftp server lives). Relaunching REBOL and attempting the same read ftp://... command as before now gets me a response of
>> foo: read ftp://xxuseridxx:[password--1--2--3--4]/path/file.txt
** Script Error: Cannot use write on this type port. ** Where: write-io sub-port #{05020002} 4 read-io
>>
Hmmm... So maybe wild-carding doesn't work on dotted quads??? Next attempt: Go back to user.r and insert the lines system/schemes/ftp/proxy/host: false system/schemes/ftp/proxy/port-id: false system/schemes/ftp/proxy/type: false after the set-net call. Try to run script. IT WORKS! Next attempt: Assuming that my earlier problem was trying to use a wildcard in a dotted-quad host identifier, I commented out the three lines above and changed the proxy bypass setting per system/schemes/default/proxy/bypass: ["1.2.3.4" "*.fedex.com"] (where, of course, the read address appeared instead of the bogus class A address above). Again, I get a response of ** Script Error: Cannot use write on this type port. ** Where: write-io sub-port #{05020002} 4 read-io So, I'm now inclined to conclude that dotted quat host identifiers (as string data) are not acceptable in the .../proxy/bypass list. After re-inserting the .../ftp/proxy/... lines above (between the set-net call and the .../proxy/bypass: setting) I get the transfer to work correctly. All of the above experimentation has been done on the HP-UX box. After inserting the .../ftp/proxy/...: false lines into user.r on my desktop Linux box, it too is able to connect to the ftp server that is the object of my afflictions. QUESTIONS: I'm still concerned about some things here: 1) Why does a URL of the form ftp://user:[pass--host]/path/file.ext work for Netscape, but not for REBOL? I'll try to track down some SquidVolk on our side and find out what they changed before following up on this question. 2) Is "1.2.*" supposed to work in a .../proxy/bypass list? What about "1.2.3.4"? I know it's lame that the sysadmin of the ftp server gave me a dotted quad instead of a host name, but that IS a valid way to identify a host. These strike me as legimate bugs, unless someone can point out something dumb that I did (or should have done). Thanks! -jn- [holger--rebol--com] wrote: