tcp ports and buffer size? Holger?
[1/7] from: jean:holzammer:faedv-n:bayern at: 11-Sep-2001 7:35
> - why 'copy doesn't block if I don't use /no-wait refinement?
copy will only wait if it still expects data to be transmitted.
Maybe the web server cancels sending data when a specific amount of
bytes of a specific file has been sent even if the file is not sent complete
?
I experienced this several times when downloading files via a web
browser. ALways with the same files.
Imagine the web server requests the data to be sent from another web
server and gets a timeout while downloading. Then it will stop transmitting
to your client , too.
Jean
[2/7] from: petr:krenzelok:trz:cz at: 30-Aug-2001 12:27
Hello,
today I needed to increase number of our downloadable mainframe files.
This time - first ones from SAP R3. Not much of a problem, unix-like ftp
we don't have access too, so I have to wrap company website using post
method to get files.
However, download of two of my files ended with 4096 bytes of downloaded
data. I wondered why and found out some strange behavior I don't
understand. I have following code:
http-port: open [
scheme: 'tcp
port-id: port-spec/port-id
timeout: port-spec/timeout
host: port-spec/host
user: port-spec/user
pass: port-spec/pass
]
;inserts 'post request
insert http-port http-request
;uncomment the next line if you wish to skip some amount of info
;http-port: skip http-port 2000
view/new layout [
styles styly
vh2 gold "Stahuji soubor, čekejte prosím ..."
across
lab "Staženo: " p: text 80 bold right white "0" return
lab "Čas:" t: text 80 bold right white "0"
]
l: 0
start: now/time
while [data: copy/part http-port buffer-size][
print mold data
append target-port data
l: l + (length? data)
p/text: to-string l
t/text: to-string now/time - start
show p show t
]
print type? data
close http-port
close target-port
So, basically, I used pure 'open - no other refinement. No matter if I
used pure while [data: copy http-port], I always ended with file of 4096
bytes size. I even tried to use "wait http-port" inside of 'while block,
but no success - "print type? data" following the loop always reported
'none. I also tried to use /no-wait refinement, but still the same.
I have buffer-size defined 20000. Now I lovered buffer size to be less
than those mystery 4096 bytes, to 2000, and my file was downloaded
correctly. So, what am I doing wrong?
- why 'copy doesn't block if I don't use /no-wait refinement?
- why using /no-wait and while [data: copy http-port] copies those 4096
bytes, then leaves loop, reporting data being none!, while it should be
an empty string instead? (waiting for port doesn't help here, loop is
left anyway ...)
My temporal solution is to use low enough buffer size and copy/part
http-port buf-size ....
Thanks a lot,
-pekr-
[3/7] from: jean:holzammer:faedv-n:bayern at: 30-Aug-2001 13:55
> - why 'copy doesn't block if I don't use /no-wait refinement?
copy will only wait if it still expects data to be transmitted.
Maybe the web server cancels sending data when a specific amount of
bytes of a specific file has been sent even if the file is not sent complete
?
I experienced this several times when downloading files via a web
browser. ALways with the same files.
Imagine the web server requests the data to be sent from another web
server and gets a timeout while downloading. Then it will stop transmitting
to your client , too.
Jean
[4/7] from: twatkins:datajunction at: 30-Aug-2001 9:12
I am not a Rebol expert, but I would suspect that it may have
something to do with TCP/IP packet sizes.
If you receive a packet smaller than your buffer, Rebol sees that
there was not enough data to fill the buffer, and so might assume
that the transfer is complete.
If you receive a packet larger than your buffer, Rebol is probably
smart enough to detect that there is still more data waiting to be
received, and so will continue receiving until it cannot fill the
buffer.
Just out of curiosity, why is the term Rebolish used in stead of
Rebollious?
To Rebel is to be Rebellious, and so it seems to me that to Rebol,
is to be Rebollious
Just a thought.
-Terwin
----- Original Message -----
From: "Petr Krenzelok" <[Petr--Krenzelok--trz--cz]>
To: <[rebol-list--rebol--com]>
Sent: Thursday, August 30, 2001 5:27 AM
Subject: [REBOL] tcp ports and buffer size? Holger?
Hello,
today I needed to increase number of our downloadable mainframe
files.
This time - first ones from SAP R3. Not much of a problem, unix-like
ftp
we don't have access too, so I have to wrap company website using
post
method to get files.
However, download of two of my files ended with 4096 bytes of
downloaded
data. I wondered why and found out some strange behavior I don't
understand. I have following code:
http-port: open [
scheme: 'tcp
port-id: port-spec/port-id
timeout: port-spec/timeout
host: port-spec/host
user: port-spec/user
pass: port-spec/pass
]
;inserts 'post request
insert http-port http-request
;uncomment the next line if you wish to skip some amount of
info
;http-port: skip http-port 2000
view/new layout [
styles styly
vh2 gold "Stahuji soubor, čekejte prosím ..."
across
lab "Staženo: " p: text 80 bold right white "0" return
lab "Čas:" t: text 80 bold right white "0"
]
l: 0
start: now/time
while [data: copy/part http-port buffer-size][
print mold data
append target-port data
l: l + (length? data)
p/text: to-string l
t/text: to-string now/time - start
show p show t
]
print type? data
close http-port
close target-port
So, basically, I used pure 'open - no other refinement. No matter if
I
used pure while [data: copy http-port], I always ended with file of
4096
bytes size. I even tried to use "wait http-port" inside of 'while
block,
but no success - "print type? data" following the loop always
reported
'none. I also tried to use /no-wait refinement, but still the same.
I have buffer-size defined 20000. Now I lovered buffer size to be
less
than those mystery 4096 bytes, to 2000, and my file was downloaded
correctly. So, what am I doing wrong?
- why 'copy doesn't block if I don't use /no-wait refinement?
- why using /no-wait and while [data: copy http-port] copies those
4096
bytes, then leaves loop, reporting data being none!, while it should
be
an empty string instead? (waiting for port doesn't help here, loop
is
left anyway ...)
My temporal solution is to use low enough buffer size and copy/part
http-port buf-size ....
Thanks a lot,
-pekr-
[5/7] from: petr:krenzelok:trz:cz at: 30-Aug-2001 18:02
----- Original Message -----
From: "Travis Watkins" <[twatkins--datajunction--com]>
To: <[rebol-list--rebol--com]>
Sent: Thursday, August 30, 2001 4:12 PM
Subject: [REBOL] Re: tcp ports and buffer size? Holger?
> I am not a Rebol expert, but I would suspect that it may have
> something to do with TCP/IP packet sizes.
>
> If you receive a packet smaller than your buffer, Rebol sees that
> there was not enough data to fill the buffer, and so might assume
> that the transfer is complete.
I am not sure and if so, that would be the explanation. But how could I know
what packet size opposite side is going to provide me with? :-)
Well, I think that if Rebol defined buffer size is larger than packet
received, behavior of Rebol depends upon the /no-wait refinement:
- if /no-wait is used, rebol simply reads what is available, now matter if
you will end-up with an empty string, or some string shorter than defined
buffer size. You simply check for 'none, which means end of transmittion.
- on the other hand - if /no-wait is not used, and your buffer size is
larger than data received, you should end-up with blocking 'copy function
behavior, waiting at the port for another data to arrive.
But none of above works for me, that's the reason I wonder what is happening
... On the other hand - server on the other side is some strange IBM Rexx
stuff, so who knows :-)
Thanks for your input anyway ...
Cheers,
-pekr-
[6/7] from: holger:rebol at: 30-Aug-2001 9:48
On Thu, Aug 30, 2001 at 09:12:16AM -0500, Travis Watkins wrote:
> I am not a Rebol expert, but I would suspect that it may have
> something to do with TCP/IP packet sizes.
>
> If you receive a packet smaller than your buffer, Rebol sees that
> there was not enough data to fill the buffer, and so might assume
> that the transfer is complete.
No, these things are handled by the TCP layer, not by REBOL.
The problem is most likely that the web server prematurely closes the
connection, for whatever reason. That is not all that uncommon. Happens
all the time in web browsers and results in missing images or backgrounds.
Petr, try using open/direct instead of a plain open.
--
Holger Kruse
[holger--rebol--com]
[7/7] from: petr:krenzelok:trz:cz at: 30-Aug-2001 20:57
----- Original Message -----
From: "Holger Kruse" <[holger--rebol--com]>
To: <[rebol-list--rebol--com]>
Sent: Thursday, August 30, 2001 6:48 PM
Subject: [REBOL] Re: tcp ports and buffer size? Holger?
> On Thu, Aug 30, 2001 at 09:12:16AM -0500, Travis Watkins wrote:
> > I am not a Rebol expert, but I would suspect that it may have
<<quoted lines omitted: 8>>
> all the time in web browsers and results in missing images or backgrounds.
> Petr, try using open/direct instead of a plain open.
Yes, I tried it too. Btw: just few days ago Gabriele told me 'direct doesn't
apply to tcp ports, but maybe I am just confused in what he meant.
btw: you are probably right it can be server dependent issues, as our three
new data files we need to download are from another server, while our files
from the first server we receive data from handles 20KB data flawlessly.
btw: with upcoming rebol releases - are we going to see any changes/addition
to port mechanisms? (Now I don't mean async stuff, but mysterious, record
based port type you mentioned some time ago ...)
Thanks for your help anyway ...
-pekr-
Notes
- Quoted lines have been omitted from some messages.
View the message alone to see the lines that have been omitted