World: r3wp

Kaj: "Cheyenne doesn't seem to load its MIME definitions from the 
standard location on Unix systems, %/etc/mime.types, loading an internal 
list %misc/mime.types instead"


Yes, it was done that way to ensure a common base of mime-types across 
all OSes. Anyway, I have in my todo list an entry for such a feature. 
I'm not yet sure if it should be by default, or controled by an explicit 
option.

Kaj: "From earlier messages here, it was my understanding that Cheyenne 
on Linux can run its UniServe taskmaster processes under a non-root 
user account by enabling 'userdir in the modules and 'user and 'group 
in the globals. However, when I do that, all of Cheyenne is still 
running as root. The designated user and group exist. What else do 
I need to do?"


It was working last time I've checked. What Cheyenne version are 
you using? Have you remove all the log files generated by Cheyenne 
that are own by root before testing? Did you tried with other combinations 
of user and group?

Terry: short anwser: yes and no. Long answer: you should be more 
precise in your question. I guess that you're asking if a *single* 
Cheyenne instance can handle 10k concurrent connections on long lasting 
communications (HTTP keepalive or web sockets), the answer is : I 
don't know, I never tested that case. My guess is that if all clients 
are not sending requests simultaneously, Cheyenne should be able 
to handle it, but with a probably significant latency in the responses. 
For a real world web site, when dealing with short lasting requests, 
having a continuous load of 10k connections would mean that you're 
having a few million hits per hour making your web site one of the 
top 50 sites in the world, giving you enough money to invest in a 
huge cluster of servers to nicely handle the load with dozens of 
Cheyenne's instances ;-).

Can I cluster Cheyenne?

Just kidding :)

has anyone tried putting a webcache like Varnish in front of Cheyenne?

I'm only concerned with latency with 100 clients... my current system 
uses SQL and bogs down nastily while I do analysis on 4000 items 
and their properties (sorting, filtering, finding etc)
I/O Costs:

L1: 3 cycles
L2: 14 cycles
RAM: 250 cycles
DISK: 41,000,000 cyles
NETWORK: 240,000,000 cycles

Doc, you should edit this wikipedia article as well 

http://en.wikipedia.org/wiki/Comparison_of_lightweight_web_servers

You can dispatch load between several Cheyenne instances very easily 
using tools like nginx.

Lightweight web servers are Web servers which have been designed 
to run with very small resource overhead because of hardware, environment, 
or simply for the challenge of it.


According to this definition and the example of target devices (Calculator, 
C64, 64KB devices), I don't think that Cheyenne qualifies due to 
REBOL's quite large memory footprint.

Terry, why don't you add Cheyenne to that Wikipedia page? A project 
is not supposed to comment on its own product on Wikipedia

There are several servers in there that are not tiny weight, pretty 
much in the ball park of Cheyenne, if you were look at code size 
and handling

Graham: a few links about your question: 

http://en.wikipedia.org/wiki/Transport_Layer_Security#Support_for_name-based_virtual_servers
http://en.wikipedia.org/wiki/Server_Name_Indication

SNI is in stunnel's todo list: http://stunnel.mirt.net/?page=todo_sdf

It also says that NginX supports it, so that may be a better option 
than STunnel

Definitive information!  Great ... I shall look at NginX

http://www.nginx.org/en/docs/http/configuring_https_servers.html

at the bottom talks about SNI support

Anyone got some quick instructions on how to use Nginx as a proxy 
for Cheyenne?

I use something like this in my nginx config:
        location ~ \.(rsp|cgi)$ {
            proxy_pass   http://lucya.desajn.web:8080;
            
            proxy_set_header   X-Real-IP        $remote_addr;

            proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
            proxy_set_header   X-URI $request_uri;

            proxy_set_header   if-modified-since $http_if_modified_since;
            
			client_max_body_size       10m;
            client_body_buffer_size    128k;

            proxy_connect_timeout      90;
            proxy_send_timeout         90;
            proxy_read_timeout         90;

            proxy_buffer_size          4k;
            proxy_buffers              4 32k;
            proxy_busy_buffers_size    64k;
            proxy_temp_file_write_size 64k;
        }

I use nginx as a frontend server and cheyenne as a backend where 
Cheyenne ca cretate static pages served by nginx... My motto for 
thic combo is: where west meets east as its American-Russian connection:)

Thanks... now anyone setup https with SNI ?  :)

I never tryied... but maybe you can try something like that http://kbeezie.com/view/configuring-sni-with-nginx/

ok .. thanks

Oldes: would it help if X-Real-IP header was used by Cheyenne to 
store the real client IP in HTTP logs? I was thinking about addind 
an option to tell Cheyenne that a HTTP header is carrying the real 
IP.

I get an "confirm" error from a RSP page that tries to delete a local 
file. How can I do this?

What do you get when trying to delete this file from console launched 
with the same uid/gid than Cheyenne?

I start cheyenne as root (not recommended I know but...). So this 
shouldn't be a problem.

I have the feeling that the security dialog is jumping in... but 
I use the Linux SDK version.

Are you using your own encapped Cheyenne binary?

I found a series of problems due to which switching to a non-root 
user and group doesn't work

The primary reason was that, although the 'library interface component 
was supposed to be included in R2.7.7, it is in View but not in Core, 
at least in the Linux version

This makes Cheyenne run with a primitive configuration without an 
advanced system interface

I asked in RebDev to fix this in R2.7.8

Trying with View only works when the window environment is started, 
even when X11 is installed on a machine, so this is game over for 
headless servers

The to do list for 2.7.8 mentions fixing the related error message, 
so I suspect Carl has found that this doesn't work when he briefly 
tried to run Cheyenne on his new server

At least it can be tested with X11 running. Then I find that Cheyenne 
searches for the GNU C library in several places, but not in the 
place where Syllable Server has it. Here's a patch to misc/unix.r 
that fixes this for both Syllable and GoboLinux (the new version 
that's currently in development):

--- unix.r.original	2010-05-09 03:22:33.000000000 +0200
+++ unix.r	2010-05-10 00:36:14.000000000 +0200
@@ -5,6 +5,8 @@
 		exists? libc: %libc.so.6
 		exists? libc: %/lib32/libc.so.6
 		exists? libc: %/lib/libc.so.6

+		exists? libc: %/System/Index/lib/libc.so.6  ; GoboLinux package

+		exists? libc: %/system/index/framework/libraries/libc.so.6  ; 
Syllable
 		exists? libc: %/lib/libc.so.5
 	]
 	libc: load/library libc

After that I found several more problems that make user switching 
unusable. Eventually I concluded that a rewrite of mod-userdir is 
necessary. I'm working on that and will post a patch after testing

Kaj: thanks, this mode has never been deeply tested. I'm applying 
your patch in svn right now.

Doc, no I'm using cheyenne0920 version (maybe quite old but it works).

Cheyenne binaries are encapped with [secure none] header to avoid 
such issues. I wonder if the LAUNCH native used to fork worker processes 
is correctly transmitting boot flags to child processes.

You should try encapping latest version from SVN with your SDK. Let 
me know if it fixes or not your issue.

Here's my patch to mods/mod-userdir.r:

--- mod-userdir.r.original	2010-05-09 19:28:10.000000000 +0200
+++ mod-userdir.r	2010-05-11 00:45:24.000000000 +0200
@@ -12,40 +12,81 @@
 	on-started: does [do boot-code]
 	on-reload:  does [clear boot-code]
 	
-	get-ugid: func [name [string!] /local file uid gid][
-		if none? attempt [file: read %/etc/passwd][
+	get-ugid: func [name [string!] /local file line uid gid][
+		unless attempt [file: read/lines %/etc/passwd][
 			log/error "accessing /etc/passwd failed"
 			return none
 		]
-		unless parse/all file [
-			thru name 2 [thru col]
-			copy uid to col skip
-			copy gid to col
-			to end
-		][
-			log/error "reading /etc/passwd failed"
+		foreach line file [
+			if all [line: find/case/match line name  col = first line][
+				return either parse/all next line [
+					thru col
+					copy uid to col skip
+					copy gid to col
+					to end
+				][
+					reduce [to-integer uid to-integer gid]
+				][
+					log/error "invalid format reading /etc/passwd !"
+					none
+				]
+			]
+		]
+		log/error "user not found in /etc/passwd"
+		none
+	]
+	
+	get-gid: func [name [string!] /local file line gid][
+		unless attempt [file: read/lines %/etc/group][
+			log/error "accessing /etc/group failed"
 			return none
 		]
-		reduce [to-integer uid to-integer gid]
+		foreach line file [
+			if all [line: find/case/match line name  col = first line][
+				return either parse/all next line [
+					thru col
+					copy gid to col
+					to end
+				][
+					to-integer gid
+				][
+					log/error "invalid format reading /etc/group !"
+					none
+				]
+			]
+		]
+		log/error "group not found in /etc/group"
+		none
 	]
 	
-	change-id: func [id [word! integer!] /user /group][
-		if word? id [
-			if none? id: get-ugid mold id [return none]
-			id: pick id to-logic user

+	change-id: func [id [string! integer!] /user /group /local gid][
+		either string? id [
+			unless id: get-ugid id [return none]
+			set [id gid] id
+		][
+			gid: id
 		]
-		either user [
+		if group [setgid gid]
+		if user [
 			;logger/file.log: join logger/file ["-" id %.log]
 			setuid id
-		][setgid id]
+		]
+	]
+	
+	change-gid: func [id [string! integer!]][
+		if string? id [
+			unless id: get-gid id [return none]
+		]
+		setgid id
 	]
 	
 	words: [
 		user: [word! | integer!] in globals do [
-			repend boot-code ['change-id/user to-lit-word args/1]

+			repend boot-code either word? args/1 [['change-id/user/group 
mold args/1]] [['change-id/user args/1]]
 		]
 		group: [word! | integer!] in globals do [
-			repend boot-code ['change-id/group to-lit-word args/1]
+			unless empty? boot-code [change boot-code [change-id/user]]

+			insert boot-code reduce ['change-gid either word? args/1 [mold 
args/1][args/1]]
 		]
 	]
 ]
\ No newline at end of file

The biggest problem was that it was switching the user and then the 
group - but after switching the user away from root, you have just 
given away your right to switch your group. So this never worked, 
and with the processes still having the group root, they could still 
access almost everything on a system, so there was hardly improved 
security

This is hard to see in the standard process list on Linux, because 
you have to use ps u -G <group>. These errors are also not logged

The new code reverses this order to make it work

IDs that you specified were looked up in the user file - also if 
you specified the group. Although in most cases on most systems, 
user names have a matching group name with the same number, this 
is clearly wrong

Specifying just the user already looked up the group number of the 
user, but this was not used for the group setting. As said above, 
it's not very meaningful to only change the user ID away from root, 
so now specifying only a user changes both the user ID and the group 
ID to the IDs corresponding to the user

I'm not sure how meaningful it is to also specify a separate group 
(different from the group of the user), but if you do it should be 
looked up in the group file instead of the users file, so now it 
does that

The users and groups files are line oriented, but they were searched 
as a whole with PARSE. This can easily go wrong, for example if the 
name appears elsewhere in the file, for example in the comment field. 
They're now searched line by line