ANN: Rugby configurable (with compression)

Hi! just a few questions: - I can read in the doc, that max no of threads is 100? Or just simply - what happens to new connections, if max number of threads is hit? - are my following thoughts correct? I want to build following scenario (will soon do so, once I get my new comp running) - Let's say I have web server (Apache) + FastCGI in external mode, which just connects to listening rebol process. Let's call such Rebol process main-server. - This main-server then accepts the query, and has three or more, local or remote, Rugby powered, additional child servers available, so it can do simple load balancing. Apache (FastCGI protocol) | | main-server (Rebol proxy = multiplexed listen mode + Rugby client) | ------------------------------------------------ | | | child1 child2 child3 | | | ODBC ODBC ODBC | | | ------------------------------------------------- | RDBMS I hope my drawing makes it easier to understand (I just hope my email client wraps it correctly :-) Now the question/brainstorming: - Main-server uses chaining to call to one of three (or more) child servers, so it acts as proxy. Each of child servers is allowed to maintain e.g. 20 simultaneous connections to database, or just accept 20 request from main-server. Each database query response is e.g. 20 KB, formatted http output. If we will send responses from all three childs in parts, it means 60 responses at a time, each of 20 KB, it means 1.2 MB - not that bad. Let's just say it is a peak on our 10 Mbit line. But - as for webserver responses, it is a well known issue, that it is better to send request once complete, at a time, not just in parts - it is much faster. So, - should also child1,2,3 send responses one at a time, or just in parts, as they build various phases of html page? Or does it even really matter? - Question no 2 - main-server, our proxy, needs to use some listening loop, store pairs of connections, do some primitive load balancing, etc. So I need something like Rugby server, without actually Rugby server message format :-) Well, - Imagine I want to provide above described mechanism to other app developers - Delphi, Visual Objets, etc. - they surely have some kind of socket or eve http classes available - so - going thru Apache is one more step. What about to allow them to connet directly to my proxy? What protocol should I use to communicate with such apps? Is it good to stay with http, and some part of GET or POST method, holding encapsulated query from above mentioned Delphi, etc. systems? Thanks for helping me to set my test environment. If I will be succesfull with my tests, it will be time to build "Renata", visual configurable Rugby based server, where you can select number of child servers, max requests per server, etc. Time to prove even women can play Rugby ;-) Cheers, -pekr-

Well, another level I suggested to Maarten for Rugby was - Rebol 'cluster - transparent propagation of functionality. Rebol clustering servers would just exchange exposed functionality, so call to any of them would do the work, but well, it needs probably some deep thinking before implementation .... -pekr-

The last one. If there are more than max-threads waiting, threads are processed despite network activity. So you can either choose for threading (set this number low, i.e. 1 or 5 or so) or network serving, set this very high (i.e. 1000+)

This looks good.

First "chaining" is implemented in servers. To use it in main-server you'd have to use /deferred and check with get-result in the event loop. But that should be easy. Then it is easy: child1..3 just do local Rebol code generating HTML from one ODBC connection. Remember that only one thing happens at once, so there is no need opening 20 connections here. Unless you use the Rugby threading API, but I think you should not for this. If child# is done, it transparently returns the result to main-server, where main-server picks it up using get-result. main-server then prints the HTML out, and you are done.

I'd use the fastcgi like loop as in the REBOL docs, read the params, make the request, store the id. If you do a wait with a timeout you can iterate every x millisec. over your list with ids to see which are ready to be sent back. Don't forget to store the fastcgi connection and the Rugby id in a 1-1 mapping. You may look at the main loop in hipe.r in Rugby, which does exactly this for TCP.

That would be fun!