World: r3wp

In your Uniserve service, build a list of ports with every client 
connecting (on-new-client event). When required, walk through the 
list of ports and broadcast whatever you want to the selected ones 
(or everyone). See this Chat application server-side source as an 
example of how to achieve that (it's not an Uniserve service, but 
it's very close anyway) : http://cheyenne-server.googlecode.com/svn/trunk/Cheyenne/www/ws-apps/chat.r


The resulting chat application is here : http://demo.cheyenne-server.org:8080/chat.html

One question Doc , I know you invested a tons of time info figuring 
out all thinks needed to make async sockets with rebol work so well, 
but..  did you ever consider using something like libevent ( http://www.monkey.org/~provos/libevent/
) or libev ( http://software.schmorp.de/pkg/libev.html) . These 
libraries are very popular with embeding in many languages ( and 
show outstanding benchmarks ) last few years after the C10K problem 
was formulated ( http://www.kegel.com/c10k.html)

I am asking just out of curiosity and because it could mean aditional 
benefits for uniserve (which I am looking to use for sometihng now) 
and by that of course also cheyenne

Libevent were suggested for R3, to be incorporated, but Carl said, 
that event model was rewritten for R3, and that it is really good. 
IMO Uniserve is relying on R2 even mechanism, I am not sure how it 
could use libevent instead? But that is for gurus to consider ..

I'm aware of libevent. Wrapping such lib in R2 would mean, at least, 
giving up on REBOL ports and REBOL's event loop. Quite a huge price 
to pay (UniServe couldn't be used with View apps, nor could receive 
system:// port events anymore). There's also the need to call REBOL 
function from the C side, which is not well supported in R2 (not 
even in R3...yet).

I also don't know. I suppose it would mean to change the rebol event/async 
handling with this. I know this would be a huge decision so I am 
not expecting any answer or anything.. just wanted to put into discussion

Aha .. cool ..

It should be possible to integrate it in R3 by rewriting the network 
part of the host kit (if both licenses are compatible).

while I am throwing in these libraries .. have you heard for mongrel 
http parser ?

I've heard of Mongrel server. What's special with its http parser?

mongrel is a server for ruby .. but it has a open sourced http parser 
that is written in "ragel" which is engine that makes some fast and 
lightweigh C code out of state machine specification. it's known 
as to be very fast and very robust because of that ( I listened too 
much online video talks in my life ) ... I will try to find some 
links

http://www.complang.org/ragel/

http://www.zedshaw.com/essays/ragel_state_charts.html

Ah, thanks for the info, I'll check the parsing rules used (that's 
a real PIA to get it right *and* secure). For the speed concern, 
a PARSE-based solution shouldn't be much slower than a C parser.

http://vision-media.ca/resources/misc/ragel-examples-- look here 
.. maybe this is the specification find "Mongrel HTTP 1/1"

ha , this is similar as more advanced rebol parse :)

I mean more advanced usage of rebol's parse

yes, I imagine http to get fully officially right is a major pain

At first look, Ragel relies on callbacks on matched patterns, so 
not a good fit for R2.

aja.. didn't look that far .. I thougth it parses given input and 
gives some result , but because result is complex callbacks might 
be nevesarry ..

**if** you found it's worth it you could make callbacks in c .. collect 
the result and only return it to rebol as data

but for the starters it requieres worka and time to discover if it's 
even worth it

I saw some huge graph one time as "whole HTTP spec" .. it was huge, 
and worying how complex it looked .. I will try to find it

I think that parse is powerfull enough to handle such parsing task. 
But finding common patterns out there might be helpful ...

Yes, surely parse can do it... I am just debating .. I am not sure 
if mongrell is really that awesome.


I was thinking that speedwise the upper bound of the http server 
is determined by socket handling and http parsing probably? Meaning 
that even if you have everything in ram and prepared you can't serve 
more thatn that. Cheyenne has a *very* high upper bound for a dynamic 
language (I was many times expressing my surprise and getting 250 
req/s was the reason I returned back to rebol with doing all webapps 
in it now).

for example of mongrell .. you typically run multiple (like 5) mongrell 
servers with ruby behind a nginx.. cheyenne does all this by itself 
so cheyenne also uses multicore /cpu better than other dyn lang servers 
by default. I am just thinking outloud if there are any prospects 
to make cheyenne/uniserve even more blazing if needed in future.

this is manually improved ragel generated http parser http://four.livejournal.com/1033160.html

uses callbacks too

hm.. I can't find that http diagram

aha .. http://webmachine.basho.com/diagram.html.. it's horibble 
basically .. if I would create a webserver I would just make 200 
and few errors, f* with the rest :)

Thanks for the diagram, there's probably a few branches that I could 
add to Cheyenne for better HTTP standard support.

Improving Cheyenne/UniServe: adding multithreading could make it 
scale much higher with much lower memory footprint. Currently, the 
main process stabilizes around ~20MB after a few hundred requests 
and each worker process take ~15-20MB depending on the application 
and loaded 3rd-party librairies. So for a server script what would 
take 1s to complete, supporting 100 clients simultaneously would 
require today ~2GB of memory. This is huge. 


Carl stated recently that threads overhead is 1MB, so with multithreading 
support, the memory usage for such use case would drop to ~100MB, 
which is an order of magnitude lower (not mentioning the speed gains 
and code simplifications resulting from dropping TCP-based IPC).

That's the main improvement that could be made internally in Cheyenne. 
Other external improvements could be to use a fast load balancer 
like nginx dispatching requests over several Cheyenne instances (over 
multiple machines), or building more advanced clusters where each 
node would be a Cheyenne server with session exchanging abilities 
between nodes.

For the "IRC" client, I'm not clear on how to build the list of ports 
in Uniserve for all connected users. On the on-new-client function, 
do I do something like append shared/client-list self?

Looks like I should be doing append shared/client-list client instead.

Briefly, can someone tell me the difference between Uniserve and 
Rebol/Services ?

R/S is a network protocol, mostly used for RPC. Uniserve is an infrastructure 
you can use to build network protocols on. For instance, the Cheyenne 
web server is built on Uniserve. For that matter, at one point someone 
built an R/S implementation on Uniserve.

nve - R/S were never properly finished. And for R3, Carl has lighter 
network services in mind. I would not bet on R/S for now ...

ok thx, I'm going to go deeper in Uniserve.

As Brian says, they're largely complementary. UniServe is mostly 
about the transport layer, while RebServices is mostly about the 
messaging layer

I wonder how it would work to partner my smtp ( receiving server 
) with the smtp ( sending ) functionality available in Cheyenne

Only one way to find out

My frustration with the iphone is the cause of this enquiry

Graham: merging both smtp servers codebase could be a good option 
and a way for Cheyenne to get 7-bit SMTP server support (which it 
is lacking currently, so classified as still "experimental").

I've started using UniServe in a production, should I use the one 
comes with Cheyenne or UniServe-r099?

The one from Cheyenne package.

No official documentation, but change-log is enough I think, right? 
(And services/protocols in Cheyenne as examples)

Exactly. You can also use the documentation from http://softinnov.org/rebol/uniserve.shtml

And for new features, have a look in Cheyenne's changelog.

Thanks a lot. I might have some questions while coding. I'm writting 
a SMS Send/Receive/Forward server & client.