r3wp [groups: 83 posts: 189283]
  • Home
  • Script library
  • AltME Archive
  • Mailing list
  • Articles Index
  • Site search
 

World: r3wp

[!Cheyenne] Discussions about the Cheyenne Web Server

Will
21-May-2008
[2585x3]
impressed! 8) I finally gave another try at php support in cheyenne 
and after patching fastcgi.c as suggested it now works like a charm.

If you are on os x and use macports, here is a way to patch and compile:

sudo port install php5 +mysql5 +fastcgi
sudo port uninstall php5
cd /opt/local/var/macports/distfiles/php5/
sudo tar -xjf php-5.2.6.tar.bz2
>> run patch below
tar -cjf php-5.2.6.tar.bz2 php-5.2.6
sudo port install php5 +mysql5 +fastcgi checksum.skip=yes

copy of Dockimbel's patch with path fixed for this example

;---- cut'n paste the following code in REBOL's console ----

patch-php: has [buffer pos][ target: %php-5.2.6/sapi/cgi/fastcgi.c 
if none? attempt [buffer: read target][ print "unable to find the 
file to patch!!" exit ] either parse buffer [ thru "int fcgi_accept_request(" 
to "if (req->fd >= 0) {" pos: to end ][ insert pos "^/^-^-^-^-break;^/^-^-^-^-" 
write target buffer print "patch applied." ][ print "failed to locate 
the line to patch!!" ] ]
patch-php ;---- end of code ----
tar -cjf php-5.2.6.tar.bz2 php-5.2.6
should be:
sudo tar -cjf php-5.2.6.tar.bz2 php-5.2.6
just in case if you want to have macports php5 support mysql 5.1.x,
install this before php5 and php5 will recognise and use it:
sudo port install mysql5-devel +server
Terry
22-May-2008
[2588]
Is that just for Mac? I was having some problems with PC
Will
22-May-2008
[2589x3]
the php source patch is needed only for non WIN platform
btw, belcome back Terry? ;-)
I just installed drupal 6.2 in a snap on Cheyenne, I really love 
the cleaning they have done on the database structure. If only it 
was in rebol...
Dockimbel
22-May-2008
[2592]
Yep the patch is for non-Windows only.
Pekr
22-May-2008
[2593]
OT: Will, do you find Drupal better than Jumla? I downloaded both, 
just haven't tried extensively yet .... from what I saw I found Joomla 
has nice interface :-)
Will
22-May-2008
[2594]
Pekr, forget joomla, that is bloated stuff, drupal is elegant, has 
good community and good documentation. I like the cleaning they have 
done with the database structure in latest version. It looks to me 
they have so much reduced the code a good php/rebol coder could translate 
it in a week.. ;-)
Pekr
22-May-2008
[2595]
It is just that I found Joomla much more user pleasant :-)
Terry
22-May-2008
[2596x2]
Doc.. have you worked with Erlang?
Will, I only come here to keep an eye on Doc's developments. And 
get sucked into the evolution group simply because I can't resist 
:)
But I've actually found post Rebol life to be very productive.
Dockimbel
22-May-2008
[2598]
Erlang, no, I just looked at the language specs and a few examples, 
why ?
Terry
22-May-2008
[2599x2]
It has some features that make it ideal for serving web pages, DB 
access etc... all the stuff that's right up your alley.
Here's a Yaws (Erlang based daemon) vs Apache benchmark.
http://www.sics.se/~joe/apachevsyaws.html

Shows Yaws handling 80,000 parallell connections.
Dockimbel
22-May-2008
[2601]
This bench looks a little odd to me. First, how can Yaws supports 
80,000 parallel connections when running on a single server (TCP/IP 
stack limit is at 65535 AFAIK) ? Second, the load is generated by 
sending request byte per byte with a delay of 10s between each bytes 
and getting in response a file of 1 byte...This isn't a web server 
real load test, it just compares the scalability of OS threads vs 
"soft threads". You don't need a benchmark to know which one scales 
better on a mono-CPU server, it's obvious. How Erlang handles multi-core 
CPU ?
Terry
22-May-2008
[2602x2]
Im not that familiar with it.. just looked interesting. 
Amazon uses Erlang for their S3 and SimpleDB products
Apaceh uses it for their CouchDB product.
Although it is possible to take advantage of multi-core technology 
by running several applications processes in parallel, the real benefit 
would be for multi-threaded applications. The challenge then is to 
change single-threaded or sparsely threaded applications into multi-threaded 
and also to debug them.

Erlang has unique properties for taking 
advantage of multi-core technology
One of the fundamental properties 
of the Erlang language is built-in support for very light-weight 
processes, each with its own memory, and using explicit message passing 
for their communication. Because of this most applications written 
in Erlang are realized as a number of cooperating Erlang processes 
representing something in the problem domain of the application, 
for example active call sessions, connections or transactions. Typically 
this will result in many thousands of simultaneously executing processes 
in a heavily loaded application.

The parallelism already present 
in most Erlang applications makes them ideal for taking advantage 
of multi-core technology, without there being any need to modify 
them.

The only thing needed is an Erlang virtual machine (VM) which 
works in a multi-threaded way, and that is what we now are releasing 
in Erlang/OTP R11B.
Dockimbel
22-May-2008
[2604x2]
Yep, cooperative multitasking built-in. Maybe we'll get that in R4 
;-)
Cheyenne, with its mono-thread full async engine, would scale very 
well on a similar benchmark, probably much better than Apache, but 
less than Yaws I guess. OTOH, a bench testing raw throughput, would 
show Apache performing quite better than Cheyenne (maybe 2 or 3 times 
better). It's hard for interpreted REBOL to compete with compiled 
C on raw power. I think that it would be possible to reduce this 
difference to 1.5 with deep optimization in Cheyenne (task scheduled 
for v1.1 according to my roadmap). R3 should open new perspectives 
for speed & memory optimizations (threading, clean async kernel, 
optimized ports, Rebcode, plugins...).
Gregg
22-May-2008
[2606x2]
Erlang has a lot going for it, but it has its own set of issues as 
well. Maarten recently spent some time evaluating it quite deeply 
for me. At this point, we're sticking with REBOL.
It is *very* attractive, all the stuff you get basically for free, 
in the context of building distributed systems.
Will
22-May-2008
[2608]
Interesting Gregg, could you say more on what pros and cons made 
you take the decision?
Graham
22-May-2008
[2609]
Where's the whitepaper?  :)
Terry
23-May-2008
[2610]
Never noticed this before.. seems someone was considering using Rebol 
as the GUI for Erlang
http://erlbol.dougedmunds.com/
Pekr
23-May-2008
[2611]
Interesting :-)
BrianH
23-May-2008
[2612]
Was? That's a new project :)
Gregg
23-May-2008
[2613]
A good starting point for doc links is http://www.erlang.org/doc.html
.


The pros are that it's been around for a long time, it was built 
to solve a specific type of problem, and has been proven to work 
for large commercial systems. It also has a nice community it seems. 


Just as C# and VB.net are capable languages, you really need to know 
the .NET framework to make things sing. Erlang, by itself, is very 
capable, but the OTP (Open Telecom Platform) provides a huge amount 
of value on top of it, if you're building distributed systems. It 
also has Yaws, Ejabberd, and other things already built that you 
can leverage.


On the downside, it's a very different model that takes some getting 
used to, though Maarten got up to speed for experimentation very 
quickly. If you're used to Prolog, that will help. It's also really 
only good for back end stuff, so we would still be doing front ends 
in something else, which wasn't the dealbreaker in our case. What 
turned us away was the security model. It's designed for use in an 
intranet type (read safe) environment, where access to machines on 
the cluster is controlled by secret cookie. If your cookie is compromised, 
they have absolute power over the node, and any nodes it shares that 
cookie with.  http://www.erlang.org/doc/reference_manual/distributed.html#11.7


We decided that, since we would end up building security on top of 
everything, using something like dialects for control, we were better 
off sticking with REBOL. There are a number of things out there already 
to bulid on (LNS, Rugby, Uniserve, BEER), we can really do things 
the way we want, in a tool we know we like and are comfortable with. 
And we know its limitations, so there will be fewer surprises.
Joe
28-May-2008
[2614]
Hi Gregg,  which is the project you're talking about  ?
Joe
29-May-2008
[2615]
Doc, the 64K limit is the limit on the number of server ports but 
not the number of connections that can be open to a single port like 
port 80
Dockimbel
29-May-2008
[2616x2]
AFAIK, every client or server connections are taking a port number 
in the 64K range. Each time a server receives a new connection it 
assigns a local port number to the client connection. That local 
port number is in the 64K range because TCP (or UDP) header storage 
space for source or destination ports are only 16bits each.
In theory, a combination of a client IP and a local port (in the 
64k range) could make a unique key allowing to handle more than 64k 
clients, but I don't think that the big 3 OS are able to exploit 
such trick in their default configuration.
Maarten
29-May-2008
[2618]
There is a paper on Philip Wadler's site: http://homepages.inf.ed.ac.uk/wadler/documents/ServiceArchitectures.pdf


Check footnote 32 on page 21 (chapter 13). That's what stopped us.
Joe
29-May-2008
[2619x2]
doc, in erlang concurrency is supported in the programming language 
(i.e. does not use the os mechanisms). I am trying to track down 
a reference were it explained you could have 2 power 32 connections 
theoretically
Maarten, yes you can set the secret cookie but if one system is compromised 
the whole distributed system is. how is that different in other architectures 
?
Dockimbel
30-May-2008
[2621]
Unless Erlang implements its own TCP/IP stack, he will hit OS limitations, 
even if, internally, he's able to handle 2^32 connections. Anyway, 
I'm interested in reading such paper if you find it.
Joe
30-May-2008
[2622x2]
not found the paper yet but here is an answer i found on the net
Pat e wrote:
> Can anyone tell me how can Yaws have almost 80,000 concurrent

> connections, and some "home-made" erlang servers over 110K concurrent
> connections if there are only 64,000 tcp/ip ports?

A TCP/IP connection is identified on each host by a
(local port, local IP, remote port, remote IP) tuple.


Therefore, there can be more than 2^16 connections to a server (even 
if

the server is listening on a fixed port) provided that they are from 
more
than one client.


In theory it's also possible to have more than 2^16 connections between 
a

specific pair of hosts provided that they use all of the available 
port

combinations. However, TCP/IP stacks generally do not support reusing

local port numbers (when the port is not specified by the client 
software)

by default, so the practical restriction in this case may still be 
2^16.

-- 
David Hopwood
Pekr
30-May-2008
[2624]
Sorry, but possible number of theoretical connection numbers is totally 
stupid reasoning for whatever kind of decision :-)
Terry
30-May-2008
[2625]
Handling 64,000+ concurrent clients is a problem I wish I had.
Dockimbel
30-May-2008
[2626x2]
Yep, in real world, there's no point accepting more than 10k connection 
on a single server. You usually hit memory and cpu limits far before 
reaching that limit. Spreading the load among multiple servers is 
the way to go. That's why the Yaws vs Apache benchmark sounds so 
theorical and irrelevant to me.
theorical => theoretical
Gregg
30-May-2008
[2628]
I like "theorical", it sounds like "the oracle". :-)
Joe
3-Jun-2008
[2629x2]
the yaws vs apache benchmark is more an example on how the erlang 
service can handle DoS
hi doc, I have a c++ backg. and seen plenty fo 30k+ concurrent but 
yes for web sites its common to have a hw redirector and multiple 
servers
Chris
4-Jun-2008
[2631]
I'm back at asking this -- how do I obtain the original request string 
-- e.g.  "/wiki/show/This_Page" within the CGI environment?  'path-info 
only gives me the translated filename (always '/qm.r' for my app)
Dockimbel
5-Jun-2008
[2632]
Chris, you should test using http://localhost/show.cgiscript it 
will display all CGI data. 'script-name should give you what you 
need.
Oldes
13-Jun-2008
[2633x2]
There must be:
	set 'NT-service? false

in misc/os.r in the block where the 'library component is not found. 
(to be able boot with Rebol/Core)
Let's say I need some functions available for all CGI scripts, where 
is the best place to init them?