Rugby / TCP woes

1) Yes, it should not be slower in any way imo! TCP connection is no magic - just raw packets on network. In reality, it should be just reverse - to set-up TCP connection, machine requesting connection sends SYN packet, remote machine sends SYN, ACK packet, acknowledging connection acceptance, then first machine once again confirms by ACK packet - so, actually setting-up tcp connection is three way process, while sending packet containing data means sending PSH, ACK with data, while other side is confirming with ACK, or PUSH, ACK, if sending data too ... or something like that ... 2) as for keeping connection "alive". I thought you are doing so with /deffered type of Rugby connection, no? The only one "problem" is - you close the port after you get-result ticket. Another problem is, that you have to explicitly poll server each n secs, if the result is already available. If I understand it correctly, you use http tunneling. Althought I don't fully understand what is going on with http tunneling, isn't it really possible to re-use already opened channel for transfer from server side, to client side? Scenario: - connection to Rugby server - server stores port in a block of port - client stores port in a block of port Is that right so far? And now - what is the problem for e.g. for chat server, to redistribute (insert-to-port) messages to each client port registered, instead of letting clients to poll the server? And btw- what does polling mean here? Is server contacted with new connection? As becuase I just looked at 'http-result-available? function, and it seems to me, that you only do 'copy on port, or is there really reconnection happening to the server?

What is more, there seems to be one strange thing happening. On czech version of W95, W98, on various set-ups, ranging from P300 to P650, the result of rugby communication is always the same - 40 - 44 sec for loop of 100 'echos. W2Kcz are OK. I am really curious, what is slowing the communication down down ... -pekr-

Exactly. Although behaviour may differ on the internet. It *is* strange. But you see that I have currently no reason to do persistent connections in Rugby.

Yes.

Yes. result-available? reads data from the client port (the *client* TCP stack) and does *not* poll the server. If it sees that it has all the data (=the return data) it closes the underlying port once you read it in the application. So: all requests use the same port for request/return. In fact, wait-for-result is just: until [result-available? index get-result index] Then the mysterious httpr (r for rugby) is just a copy of RT's http protocol that does not wait for the first two lines of the return header. Truely non-blocking. Drawback: you loose automatic http redirects and such. You get the complete http response back, so you can implement that yourself.

Yes... platform dependant behaviour. Don't get it either. --Maarten

More or less, although there are some subtle differences, e.g. in the ACK delay strategy, in buffer sizes and in the precise behavior of the Nagle algorithm. Also, the PSH flag is implemented very inconsistently across platforms. All of these issues can affect performance, in particular for asynchronous, full-duplex communication. One thing you can try is "error? try [set-modes port [no-delay: true]]". This disables Nagle and in some situations can improve performance. Don't use it for high-volume streaming though. For more information on this and other "unexplainable" performance differences between platforms, ask the Samba developers. They could tell you some stories :-). The REBOL network adaptation layer does not vary much by platform, not enough to explain those differences. It is more likely that the performance differences are the result of a problem in the script which, together with high CPU use, interacts with differences in the paging and scheduling algorithms of different operating systems, leading to different performance. Other software which combine high CPU use with a lot of I/O, e.g. compilers, often show similar performance variations.

Yes, it should, for the same input. If there are indeed differences then chances are that the input to the script (perhaps its timing) is different, and the script might have a race condition that triggers the difference in behavior. Incorrectly handled errors (e.g. from ports being closed in different order due to different timing) could explain such problems. Also, AFAIK Maarten uses the (undocumented) async-modes field in ports to implement async i/o. That field was never intended for use by anyone outside RT (anyone other than me, actually), and incorrect use may very well lead to undefined behavior, or to behavior that varies by platform. The reason why its undocumented is because it is very tricky to use correctly, in particular if you want asynchronous behavior for all situations (accepting a connection, connecting, reading, writing). "CPU eating" can easily be explained by having async-modes in the wrong state for a port that is part of system/wait-list or the argument to 'wait. In particular watch out for errors created by the other end (e.g. a closed connection), and how they are handled. An error handler which is too "global", fails to properly clean up after a port error, and leaves such a port in wait-list with async-modes in a wrong state, could easily explain busy-looping. If you have reproducable performance differences on different platforms then the best way to track down the cause is to run tcpdump. That might also reveal the reason why reusing a TCP connection slows things down. In our experience reusing TCP connections significantly improves performance, and we make use of that in Express. -- Holger Kruse [holger--rebol--com]

Aha. And I tested under high CPU load. Now I get it.

There are no errors there, otherwise I'd see them? Or do you mean race conditions internally in Rebol?

No. I use non-blocking connections, I dumped async-modes after version 1 (currently 4.3). So it has to do with using non-buffered, non-blocking connections. I noticed that not reading a buffer completely on any side of a connection may result in CPU eating on some platforms, but that is fixed now. It feels like a state machine gone mad. No proof of course ;-)

Except under high load on an Intranet. 330/sec or 80/sec between 2 1 Ghz Linux 2.4.14, Reiserfs with fast ethernet is a noticable difference. --Maarten

Holger, guru stuff. Very interesting. I ran ethereal packet monitor, but saw nothing, just some strange packets sent to DNS server. You described Rebol internals without describing them at all :-) The nicest part is the port for Holger (tm) :-) It made my day .... btw: is there any change in networking stuff in IOS already, in comparison to View for e.g.? Or will we have to wait till 3.0? -pekr-

ehm, then you do reuse already opened ports. So how does your aproach here differes from persistent connection you had in mind??? Thanks, -pekr-

== 0:00:00.982

== 0:00:00.991

== 0:00:00.951 --Gregg

** Script Error: echo expected target argument of type: file none ** Where: do-boot ** Near: echo "lala"

Enjoy!! Ammon