[CGI] web server issues
Hope you read http://www.rebol.com/docs/fastcgi.html
That REBOL doc should really answer your question. Simply put, in External mode, you do something like /path/to/script (that does not need to exist) and you direct it to certain, already running REBOL process. But - rebol has no tasking, so you have to handle accepting connections and multiplexing. It is like with Rubgy - unless you are finished, you are not available to other requests ...
Read that, but that is why the adaptive scaling with lighttpd is interesting if you put number of request/fcgi porcess on 1. Then the daemon scales for you
Maarten, I agree with your observation and you can even scale it more. If you see a web-server as just a request dispatcher to CGIs and a fast-answering-machine for user-feedback (pages, forms etc.) you just need a small and "simple" one like Cheyenne. The CGIs can be distributed to different cores (through the OS) or even to different machines (via TCP/IP).
As dispatching requests is most likely much faster than processing a request, a single web-server should serve a lot of users and a bunch of machines do the processing. This is the coarse grained multi-process approach.
With Cheyenne you can already have the main httpd process on one machine and task-handlers (RSP or whatever) on other machines 8)
I am close to autogenerating fastcgi processes, linked with Lighttpd configs and generating automagical includes that match the web server config for encap/Pro.
I am also adding a routing scheme (dialect) so basically you redirect all traffic except static stuff to a fastcgi process, it comes in the router and then it checks on extension or path (REST) what to do.
The generating part will be asking you a few questions and the generate matchhing config files and binaries that you can copy to an empty Linux box with only lighttpd installed (or enginx)
Session management is database-backed, so scaling up is hiring VPS'es and putting pound in between
With the avereage memory use fo Rebol < 10Mb you can coompute how many users I can concurrently server for complex operations (100-200 minimum), so every machine I hire can host 500 customers. That means that I should earn e0.50 customer to get a decent margin (roughly).
earn = ask
I finally learned the trick: 1) create a good interface (but w eknew that) 2) use back-end technology that lowers the cost per user (i.e. enlarges the # users per box)
Maarten, this sounds very cool. So the goal is to have a scalable web-service framwork based on fastCGI and simple tools?
That's how I should have described it in one sentence.
But yes: load-balancer -> webserver(*) -> FastCGI(*) -> MySQL
FastCGI is a rebol process with core enhancements, session mgt, RSP etc. I am also integrating autodoc from Gabriele so the files will be more "literate" and I have a module management system in place thathandles from interactive to encap.
I am using assembla.com from SVN and trac, the actual application I am building is for personal life management.
As a rebol process is only 10 Mb.... I can serve lots of users on cheap VPS's, load balance them, data backup in S3. No others invited until I get things stable enough. eed to ge things going
Sounds very cool. Go for it!!
I don't understand it a bit. I can understand webserver, fastcgi, mysql part, but what is that load-balancer part? Client side?
or some special server part?
No, before the webserver, so you scale transparantly to multiple webservers (in my scenario each webserver effectively is the load balancer for X FastCGI rebol processes; it's how nginx and lighty work)
What are the best ways to protect source code from view in a cgi script? When a script is made world-viewable, isn't that compromising the source pretty badly?
You want to set the file permissions for each script globally executable, but not globally readable. Also protect the cgi-bin folder from being read by the whole world...If you are using Apache, look at the IndexIgnore directive: http://httpd.apache.org/docs/1.3/mod/mod_autoindex.html#indexignore
One of my clients updates his site via some tool, which always seem to add some space between the lines. After some time, the page is instead of 400 rows something like 13K rows - the size goes from cca 25KB to 100KB. So I wrote a cgi script, which reads index.html and removes blank lines. Everything is OK, when I run the script from the console. But when I run it via a browser as a CGI script call, it can't write the file. Dunno why - cgi-script is being run using -cs switch, I even put secure none in there, cgi-script has the same own, grp set as index.html, but I can't write it ....
Maybe I should try the trick to connect to the FTP account and upload it there, instead of rewriting? :-)
Check the permissions of the file to be written. Are you also sure the cgi script is being executed? Check its permissions too..
I am sure, as I get cgi script output to the browser window. It just fails on the last line - write/lines %index.htm data
permissions look OK, but I will recheck. Dunno what user Apache runs cgi scripts under, so I better check it is others writeable ....
Maybe the cgi script current directory is not where the index.htm file is. See if you can list the file, eg. instead of write/lines %index.htm data do this: print mold info? %index.htm
no, it is a different directory .... the script is being run in /cgi-bin/, while index.htm is at the root of the web ...
hmm, maybe I found the problem - it has rw--r--r-- file permissions ....
Changed index.html permission to 646, and it works now. Maybe CGI scripts are being run under the Apache user or group, who knows. Thanks for pointers!
Did you try 664 first ?
no, I can try once I get at home ... that might be more secure solution, thanks ...
What is the tool your client uses which produces the extra lines? It is the source of the problem.
some old Mozilla web building SW, don't remember its name. It was some three letters name. But - maybe one of my friends is doing some small Rebol client app edits (putting news into the main site), so maybe it is his script which does so. I need to check with him, it is an encapped app, so I can't see the sources ...
Yeah, definitely check that. If it's rebol, then it can be easily fixed.
NVU is old Mozilla attempt to separate page editor into something like FrontPage. I think it is not developed anymore, but some ppl do continue with developments under the name Kompozer. More info here - https://wiki.mozilla.org/NVU:Home_Page
is there a ways to know if a rebol script is being run as cgi or as a stand-alone?
I want to print out an html error when run from cgi, but execute code when run stand-alone.
ahhh yes, I can use the cgi headers to see if they are set... that was obvious... seems, I'm still not fully recuperated from the virus which I've been "nurturing" for a few days now.
I'll have a static ip by next week so I hope that solves this issue for me..
I want to access a mysql db via a cgi interface directly with a rebol client. I have a cgi script that accesses the db and prints the results which I can read with the client but is this the best method? Is there a way to send the data directly to the client as rebol blocks or do I need to essestialy parse a cgi built web page as I am doing?
I know I can use mysql-protocol to access mysql directly but that is not hat I mean as sometimes that is not possible/allowed by some isp's.
The ISP owns the mysql db .. and opens up some tables for you to use. So, no, it's not possible unless you run your own Vm like as on Amazon or linode or slicehost or whatever.