FTP problems -- mystery.
[1/5] from: joel::neely::fedex::com at: 25-Sep-2000 14:39
I had an HP-UX server running a REBOL script in a cron job every
night to pull a data file from a wNT box. That script does
foo: read ftp://userid:[password--1--2--3--4]/path/file.txt
(wrapped in a try block), then post-processes the data before
writing it to a local file. This had been running non-stop
since mid-April.
On 26 Jun 2000 I upgraded that HP-UX box from 2.2 to 2.3
(don't try to read that aloud! ;-). Everything continued to
work properly.
On 10 Aug 2000 the above statement began consistently failing.
I can run the standard ftp utility from the command line and
retrieve the file (with almost instantaneous response), but
the REBOL script fails whether run manually or via cron.
When I run REBOL interactively and try the same command, with
tracing enabled, I get the following:
>> system/schemes/ftp/passive: on
== true
(The above inspired by Carl's note below, but the same happens
without that intervention.)
>> foo: read ftp://userid:[password--1--2--3--4]/path/file.txt
connecting to: 1.2.3.4
** User Error: Server error: tcp HTTP/1.0 408 Request Time-out.
** Where: foo: read ftp://userid:[password--1--2--3--4]/path/file.txt
>> trace/net on
>> foo: read ftp://userid:[password--1--2--3--4]/path/file.txt
URL Parse: userid password 1.2.3.4 none path/ file.txt
Net-log: ["Opening tcp for" FTP]
connecting to: 1.2.3.4
Net-log: [
none ["220" "230"]]
** User Error: Server error: tcp HTTP/1.0 408 Request Time-out.
** Where: foo: read ftp://userid:[password--1--2--3--4]/path/file.txt
>> system/schemes/ftp/passive
== true
Any suggestions from the REBOLnetworkingwizards about what I should
try next?
-jn-
[carl--rebol--com] wrote:
[2/5] from: kracik:mbox:dkm:cz at: 25-Sep-2000 22:48
Hi,
I'm not a REBOLnetworkingwizard, but from the error message
tcp HTTP/1.0 408 Request Time-out.
it seems that a HTTP, not FTP server listens at default FTP port 21,
so the WinNT server configuration may be responsible for it.
Causes of such mysterious errors are easily discovered with a packet
sniffer and analyzer, where you can see the whole TCP communication.
hope this helps,
--
Michal Kracik
[joel--neely--fedex--com] wrote:
[3/5] from: joel:neely:fedex at: 25-Sep-2000 16:20
Thanks, Michal...
But I believe the services on the host(s) are correct.
It seems that I get a similar message whenever I try to connect from
REBOL (either on the HP-UX box or my desktop box running Linux)
to ANY ftp server.
However, a transcript from a console ftp session (to the same host
I'm trying to hit from REBOL) makes it appear that a "normal" ftp
service is running on the host's port 21.
$ ftp 1.2.3.4
Connected to 1.2.3.4.
220 HOSTNAME Microsoft FTP Service (Version 4.0).
Name (1.2.3.4:user): userid
331 Password required for userid.
Password:
230-Hello, welcome to HOSTNAME.
230 User userid logged in.
Remote system type is Windows_NT.
ftp> ls
200 PORT command successful.
150 Opening ASCII mode data connection for /bin/ls.
09-25-00 07:23AM <DIR> downloads
09-24-00 06:00PM <DIR> xxxxxxx
08-24-00 09:01AM <DIR> yyyyyyy
05-12-00 03:02PM <DIR> zzzzzzz
06-30-00 05:38PM <DIR> wwwwwww
ftp> quit
221 Goodbye!
$
I get a similar response when connecting to the HP-UX server from
my Linux box, and I just checked /etc/services and /etc/inetd.conf
to verify that ftp really is running on port 21.
$ ftp my-hpux-box
Connected to my-hpux-box.
220 my-hpux-box FTP server (Version 1.7.212.2 Tue Apr 21 12:14:46
GMT 1998) ready.
Name (my-hpux-box:zzuserzz): xxuserxx
331 Password required for xxuserxx.
Password:
230 User xxuserxx logged in.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> ls
200 PORT command successful.
150 Opening ASCII mode data connection for /usr/bin/ls.
.
. long ls output suppressed here
.
226 Transfer complete.
ftp> get user.r junkfilekillme
local: junkfilekillme remote: user.r
200 PORT command successful.
150 Opening BINARY mode data connection for user.r (152 bytes).
226 Transfer complete.
152 bytes received in 0.00718 secs (21 Kbytes/sec)
ftp> quit
221 Goodbye.
$
But then, from my desktop Linux box...
>> system/version
== 2.3.0.4.2
(just FYI)
>> foo: read http://www.rebol.com/
URL Parse: none none www.rebol.com none none none
Net-log: ["Opening tcp for" HTTP]
connecting to: www.rebol.com
Net-log: {GET http://www.rebol.com/ HTTP/1.0
Accept: */*
Connection: close
User-Agent: REBOL 2.3.0.4.2
Host: www.rebol.com
}
Net-log: "HTTP/1.0 200 OK"
Net-log: ["low level read of " 11210 "bytes"]
== {<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html;CHARSET=iso-...
(OK. We've verified REBOL talking to the network.)
>> foo: read ftp://xxuseridxx:[xxpasswordxx--my-hpux-host]/
URL Parse: xxuseridxx xxpasswordxx my-hpux-host none none none
Net-log: ["Opening tcp for" FTP]
connecting to: my-hpux-host
Net-log: [
none ["220" "230"]]
** User Error: Server error: tcp HTTP/1.0 408 Request Time-out.
** Where: foo: read ftp://xxuseridxx:[xxpasswordxx--my-hpux-host]/
>>
???
-
[kracik--mbox--dkm--cz] wrote:
[4/5] from: holger:rebol at: 25-Sep-2000 15:08
On Mon, Sep 25, 2000 at 04:20:10PM -0500, [joel--neely--fedex--com] wrote:
> Thanks, Michal...
>
> But I believe the services on the host(s) are correct.
>
> It seems that I get a similar message whenever I try to connect from
> REBOL (either on the HP-UX box or my desktop box running Linux)
> to ANY ftp server.
It looks like you have configured a generic proxy server for REBOL,
and that server may no longer exist or may not be working correctly.
That also explains why all FTP connections appear to be based on
HTTP.
Please check the set-net call in your user.r file and, if you don't
need a proxy server, change "generic" to "none".
--
Holger Kruse
[holger--rebol--com]
[5/5] from: joel:neely:fedex at: 25-Sep-2000 19:43
I'm passing this status report back to the list in case someone
else gets bitten by a similar problem.
SHORT VERSION: I have enough of a clue to get a workaround going,
but still have concerns about how well REBOL and
the corporate proxy server (Squid) are getting along. I also have
what appears to be a bug in REBOL's handling of .../proxy/bypass
entries.
LONG VERSION:
That thunderous CLICK! you heard a while ago was the light going on...
I tracked this down with RickM, our intrepid sysadmin, looking over my
shoulder (credit where credit is due).
As Holger suspected, I did have a generic proxy server configured.
This had been working since mid-April, and was necessary to get 'net
traffic through our firewall (little useful things like 'upgrade!)
The server software (Squid/2.3.STABLE2) and/or its configuration
have changed in a way that is no longer compatible with REBOL. I can
still hit the desired ftp server from within Netscape by using a URL
of the form
ftp://userid:[password--1--2--3--4]/
and get a listing of the top-level ftp directory space. If I use
only the URL
ftp://1.2.3.4/
I get an error message from Squid stating (correctly) that the server
in question does not allow anonymous connections. Squid's error
message, of course, is returned as HTML.
If I disable proxy configuration for REBOL (comment out the set-net
call in user.r) the ftp connection works. Re-enabling the proxy
setting (de-commenting set-net and restarting REBOL) gets me the
timeout again.
Soooo...
Turning to Carl's draft version of REBOL Network Protocols, I tried
adding to user.r (after the set-net call) the line
system/schemes/default/proxy/bypass: ["1.2.*" "*.fedex.com"]
(where the "1.2." was actually the prefix of the class B network in
which the long-sought ftp server lives). Relaunching REBOL and
attempting the same read ftp://... command as before now gets me
a response of
>> foo: read ftp://xxuseridxx:[password--1--2--3--4]/path/file.txt
** Script Error: Cannot use write on this type port.
** Where: write-io sub-port #{05020002} 4
read-io
>>
Hmmm... So maybe wild-carding doesn't work on dotted quads???
Next attempt: Go back to user.r and insert the lines
system/schemes/ftp/proxy/host: false
system/schemes/ftp/proxy/port-id: false
system/schemes/ftp/proxy/type: false
after the set-net call. Try to run script. IT WORKS!
Next attempt: Assuming that my earlier problem was trying to use
a wildcard in a dotted-quad host identifier, I commented out the
three lines above and changed the proxy bypass setting per
system/schemes/default/proxy/bypass: ["1.2.3.4" "*.fedex.com"]
(where, of course, the read address appeared instead of the bogus
class A address above). Again, I get a response of
** Script Error: Cannot use write on this type port.
** Where: write-io sub-port #{05020002} 4
read-io
So, I'm now inclined to conclude that dotted quat host identifiers
(as string data) are not acceptable in the .../proxy/bypass list.
After re-inserting the .../ftp/proxy/... lines above (between the
set-net call and the .../proxy/bypass: setting) I get the transfer
to work correctly.
All of the above experimentation has been done on the HP-UX box.
After inserting the .../ftp/proxy/...: false lines into user.r
on my desktop Linux box, it too is able to connect to the ftp
server that is the object of my afflictions.
QUESTIONS: I'm still concerned about some things here:
1) Why does a URL of the form ftp://user:[pass--host]/path/file.ext
work for Netscape, but not for REBOL?
I'll try to track down some SquidVolk on our side and find out
what they changed before following up on this question.
2) Is "1.2.*" supposed to work in a .../proxy/bypass list? What about
"1.2.3.4"? I know it's lame that the sysadmin of the ftp server
gave me a dotted quad instead of a host name, but that IS a valid way
to identify a host. These strike me as legimate bugs, unless someone
can point out something dumb that I did (or should have done).
Thanks!
-jn-
[holger--rebol--com] wrote: