Mailing List Archive: 49091 messages
  • Home
  • Script library
  • AltME Archive
  • Mailing list
  • Articles Index
  • Site search
 

tcp ports and buffer size? Holger?

 [1/7] from: jean:holzammer:faedv-n:bayern at: 11-Sep-2001 7:35


> - why 'copy doesn't block if I don't use /no-wait refinement?
copy will only wait if it still expects data to be transmitted. Maybe the web server cancels sending data when a specific amount of bytes of a specific file has been sent even if the file is not sent complete ? I experienced this several times when downloading files via a web browser. ALways with the same files. Imagine the web server requests the data to be sent from another web server and gets a timeout while downloading. Then it will stop transmitting to your client , too. Jean

 [2/7] from: petr:krenzelok:trz:cz at: 30-Aug-2001 12:27


Hello, today I needed to increase number of our downloadable mainframe files. This time - first ones from SAP R3. Not much of a problem, unix-like ftp we don't have access too, so I have to wrap company website using post method to get files. However, download of two of my files ended with 4096 bytes of downloaded data. I wondered why and found out some strange behavior I don't understand. I have following code: http-port: open [ scheme: 'tcp port-id: port-spec/port-id timeout: port-spec/timeout host: port-spec/host user: port-spec/user pass: port-spec/pass ] ;inserts 'post request insert http-port http-request ;uncomment the next line if you wish to skip some amount of info ;http-port: skip http-port 2000 view/new layout [ styles styly vh2 gold "Stahuji soubor, čekejte prosím ..." across lab "Staženo: " p: text 80 bold right white "0" return lab "Čas:" t: text 80 bold right white "0" ] l: 0 start: now/time while [data: copy/part http-port buffer-size][ print mold data append target-port data l: l + (length? data) p/text: to-string l t/text: to-string now/time - start show p show t ] print type? data close http-port close target-port So, basically, I used pure 'open - no other refinement. No matter if I used pure while [data: copy http-port], I always ended with file of 4096 bytes size. I even tried to use "wait http-port" inside of 'while block, but no success - "print type? data" following the loop always reported 'none. I also tried to use /no-wait refinement, but still the same. I have buffer-size defined 20000. Now I lovered buffer size to be less than those mystery 4096 bytes, to 2000, and my file was downloaded correctly. So, what am I doing wrong? - why 'copy doesn't block if I don't use /no-wait refinement? - why using /no-wait and while [data: copy http-port] copies those 4096 bytes, then leaves loop, reporting data being none!, while it should be an empty string instead? (waiting for port doesn't help here, loop is left anyway ...) My temporal solution is to use low enough buffer size and copy/part http-port buf-size .... Thanks a lot, -pekr-

 [3/7] from: jean:holzammer:faedv-n:bayern at: 30-Aug-2001 13:55


> - why 'copy doesn't block if I don't use /no-wait refinement?
copy will only wait if it still expects data to be transmitted. Maybe the web server cancels sending data when a specific amount of bytes of a specific file has been sent even if the file is not sent complete ? I experienced this several times when downloading files via a web browser. ALways with the same files. Imagine the web server requests the data to be sent from another web server and gets a timeout while downloading. Then it will stop transmitting to your client , too. Jean

 [4/7] from: twatkins:datajunction at: 30-Aug-2001 9:12


I am not a Rebol expert, but I would suspect that it may have something to do with TCP/IP packet sizes. If you receive a packet smaller than your buffer, Rebol sees that there was not enough data to fill the buffer, and so might assume that the transfer is complete. If you receive a packet larger than your buffer, Rebol is probably smart enough to detect that there is still more data waiting to be received, and so will continue receiving until it cannot fill the buffer. Just out of curiosity, why is the term Rebolish used in stead of Rebollious? To Rebel is to be Rebellious, and so it seems to me that to Rebol, is to be Rebollious Just a thought. -Terwin ----- Original Message ----- From: "Petr Krenzelok" <[Petr--Krenzelok--trz--cz]> To: <[rebol-list--rebol--com]> Sent: Thursday, August 30, 2001 5:27 AM Subject: [REBOL] tcp ports and buffer size? Holger? Hello, today I needed to increase number of our downloadable mainframe files. This time - first ones from SAP R3. Not much of a problem, unix-like ftp we don't have access too, so I have to wrap company website using post method to get files. However, download of two of my files ended with 4096 bytes of downloaded data. I wondered why and found out some strange behavior I don't understand. I have following code: http-port: open [ scheme: 'tcp port-id: port-spec/port-id timeout: port-spec/timeout host: port-spec/host user: port-spec/user pass: port-spec/pass ] ;inserts 'post request insert http-port http-request ;uncomment the next line if you wish to skip some amount of info ;http-port: skip http-port 2000 view/new layout [ styles styly vh2 gold "Stahuji soubor, čekejte prosím ..." across lab "Staženo: " p: text 80 bold right white "0" return lab "Čas:" t: text 80 bold right white "0" ] l: 0 start: now/time while [data: copy/part http-port buffer-size][ print mold data append target-port data l: l + (length? data) p/text: to-string l t/text: to-string now/time - start show p show t ] print type? data close http-port close target-port So, basically, I used pure 'open - no other refinement. No matter if I used pure while [data: copy http-port], I always ended with file of 4096 bytes size. I even tried to use "wait http-port" inside of 'while block, but no success - "print type? data" following the loop always reported 'none. I also tried to use /no-wait refinement, but still the same. I have buffer-size defined 20000. Now I lovered buffer size to be less than those mystery 4096 bytes, to 2000, and my file was downloaded correctly. So, what am I doing wrong? - why 'copy doesn't block if I don't use /no-wait refinement? - why using /no-wait and while [data: copy http-port] copies those 4096 bytes, then leaves loop, reporting data being none!, while it should be an empty string instead? (waiting for port doesn't help here, loop is left anyway ...) My temporal solution is to use low enough buffer size and copy/part http-port buf-size .... Thanks a lot, -pekr-

 [5/7] from: petr:krenzelok:trz:cz at: 30-Aug-2001 18:02


----- Original Message ----- From: "Travis Watkins" <[twatkins--datajunction--com]> To: <[rebol-list--rebol--com]> Sent: Thursday, August 30, 2001 4:12 PM Subject: [REBOL] Re: tcp ports and buffer size? Holger?
> I am not a Rebol expert, but I would suspect that it may have > something to do with TCP/IP packet sizes. > > If you receive a packet smaller than your buffer, Rebol sees that > there was not enough data to fill the buffer, and so might assume > that the transfer is complete.
I am not sure and if so, that would be the explanation. But how could I know what packet size opposite side is going to provide me with? :-) Well, I think that if Rebol defined buffer size is larger than packet received, behavior of Rebol depends upon the /no-wait refinement: - if /no-wait is used, rebol simply reads what is available, now matter if you will end-up with an empty string, or some string shorter than defined buffer size. You simply check for 'none, which means end of transmittion. - on the other hand - if /no-wait is not used, and your buffer size is larger than data received, you should end-up with blocking 'copy function behavior, waiting at the port for another data to arrive. But none of above works for me, that's the reason I wonder what is happening ... On the other hand - server on the other side is some strange IBM Rexx stuff, so who knows :-) Thanks for your input anyway ... Cheers, -pekr-

 [6/7] from: holger:rebol at: 30-Aug-2001 9:48


On Thu, Aug 30, 2001 at 09:12:16AM -0500, Travis Watkins wrote:
> I am not a Rebol expert, but I would suspect that it may have > something to do with TCP/IP packet sizes. > > If you receive a packet smaller than your buffer, Rebol sees that > there was not enough data to fill the buffer, and so might assume > that the transfer is complete.
No, these things are handled by the TCP layer, not by REBOL. The problem is most likely that the web server prematurely closes the connection, for whatever reason. That is not all that uncommon. Happens all the time in web browsers and results in missing images or backgrounds. Petr, try using open/direct instead of a plain open. -- Holger Kruse [holger--rebol--com]

 [7/7] from: petr:krenzelok:trz:cz at: 30-Aug-2001 20:57


----- Original Message ----- From: "Holger Kruse" <[holger--rebol--com]> To: <[rebol-list--rebol--com]> Sent: Thursday, August 30, 2001 6:48 PM Subject: [REBOL] Re: tcp ports and buffer size? Holger?
> On Thu, Aug 30, 2001 at 09:12:16AM -0500, Travis Watkins wrote: > > I am not a Rebol expert, but I would suspect that it may have
<<quoted lines omitted: 8>>
> all the time in web browsers and results in missing images or backgrounds. > Petr, try using open/direct instead of a plain open.
Yes, I tried it too. Btw: just few days ago Gabriele told me 'direct doesn't apply to tcp ports, but maybe I am just confused in what he meant. btw: you are probably right it can be server dependent issues, as our three new data files we need to download are from another server, while our files from the first server we receive data from handles 20KB data flawlessly. btw: with upcoming rebol releases - are we going to see any changes/addition to port mechanisms? (Now I don't mean async stuff, but mysterious, record based port type you mentioned some time ago ...) Thanks for your help anyway ... -pekr-

Notes
  • Quoted lines have been omitted from some messages.
    View the message alone to see the lines that have been omitted