World: r3wp

Oldes: would it help if X-Real-IP header was used by Cheyenne to 
store the real client IP in HTTP logs? I was thinking about addind 
an option to tell Cheyenne that a HTTP header is carrying the real 
IP.

I get an "confirm" error from a RSP page that tries to delete a local 
file. How can I do this?

What do you get when trying to delete this file from console launched 
with the same uid/gid than Cheyenne?

I start cheyenne as root (not recommended I know but...). So this 
shouldn't be a problem.

I have the feeling that the security dialog is jumping in... but 
I use the Linux SDK version.

Are you using your own encapped Cheyenne binary?

I found a series of problems due to which switching to a non-root 
user and group doesn't work

The primary reason was that, although the 'library interface component 
was supposed to be included in R2.7.7, it is in View but not in Core, 
at least in the Linux version

This makes Cheyenne run with a primitive configuration without an 
advanced system interface

I asked in RebDev to fix this in R2.7.8

Trying with View only works when the window environment is started, 
even when X11 is installed on a machine, so this is game over for 
headless servers

The to do list for 2.7.8 mentions fixing the related error message, 
so I suspect Carl has found that this doesn't work when he briefly 
tried to run Cheyenne on his new server

At least it can be tested with X11 running. Then I find that Cheyenne 
searches for the GNU C library in several places, but not in the 
place where Syllable Server has it. Here's a patch to misc/unix.r 
that fixes this for both Syllable and GoboLinux (the new version 
that's currently in development):

--- unix.r.original	2010-05-09 03:22:33.000000000 +0200
+++ unix.r	2010-05-10 00:36:14.000000000 +0200
@@ -5,6 +5,8 @@
 		exists? libc: %libc.so.6
 		exists? libc: %/lib32/libc.so.6
 		exists? libc: %/lib/libc.so.6

+		exists? libc: %/System/Index/lib/libc.so.6  ; GoboLinux package

+		exists? libc: %/system/index/framework/libraries/libc.so.6  ; 
Syllable
 		exists? libc: %/lib/libc.so.5
 	]
 	libc: load/library libc

After that I found several more problems that make user switching 
unusable. Eventually I concluded that a rewrite of mod-userdir is 
necessary. I'm working on that and will post a patch after testing

Kaj: thanks, this mode has never been deeply tested. I'm applying 
your patch in svn right now.

Doc, no I'm using cheyenne0920 version (maybe quite old but it works).

Cheyenne binaries are encapped with [secure none] header to avoid 
such issues. I wonder if the LAUNCH native used to fork worker processes 
is correctly transmitting boot flags to child processes.

You should try encapping latest version from SVN with your SDK. Let 
me know if it fixes or not your issue.

Here's my patch to mods/mod-userdir.r:

--- mod-userdir.r.original	2010-05-09 19:28:10.000000000 +0200
+++ mod-userdir.r	2010-05-11 00:45:24.000000000 +0200
@@ -12,40 +12,81 @@
 	on-started: does [do boot-code]
 	on-reload:  does [clear boot-code]
 	
-	get-ugid: func [name [string!] /local file uid gid][
-		if none? attempt [file: read %/etc/passwd][
+	get-ugid: func [name [string!] /local file line uid gid][
+		unless attempt [file: read/lines %/etc/passwd][
 			log/error "accessing /etc/passwd failed"
 			return none
 		]
-		unless parse/all file [
-			thru name 2 [thru col]
-			copy uid to col skip
-			copy gid to col
-			to end
-		][
-			log/error "reading /etc/passwd failed"
+		foreach line file [
+			if all [line: find/case/match line name  col = first line][
+				return either parse/all next line [
+					thru col
+					copy uid to col skip
+					copy gid to col
+					to end
+				][
+					reduce [to-integer uid to-integer gid]
+				][
+					log/error "invalid format reading /etc/passwd !"
+					none
+				]
+			]
+		]
+		log/error "user not found in /etc/passwd"
+		none
+	]
+	
+	get-gid: func [name [string!] /local file line gid][
+		unless attempt [file: read/lines %/etc/group][
+			log/error "accessing /etc/group failed"
 			return none
 		]
-		reduce [to-integer uid to-integer gid]
+		foreach line file [
+			if all [line: find/case/match line name  col = first line][
+				return either parse/all next line [
+					thru col
+					copy gid to col
+					to end
+				][
+					to-integer gid
+				][
+					log/error "invalid format reading /etc/group !"
+					none
+				]
+			]
+		]
+		log/error "group not found in /etc/group"
+		none
 	]
 	
-	change-id: func [id [word! integer!] /user /group][
-		if word? id [
-			if none? id: get-ugid mold id [return none]
-			id: pick id to-logic user

+	change-id: func [id [string! integer!] /user /group /local gid][
+		either string? id [
+			unless id: get-ugid id [return none]
+			set [id gid] id
+		][
+			gid: id
 		]
-		either user [
+		if group [setgid gid]
+		if user [
 			;logger/file.log: join logger/file ["-" id %.log]
 			setuid id
-		][setgid id]
+		]
+	]
+	
+	change-gid: func [id [string! integer!]][
+		if string? id [
+			unless id: get-gid id [return none]
+		]
+		setgid id
 	]
 	
 	words: [
 		user: [word! | integer!] in globals do [
-			repend boot-code ['change-id/user to-lit-word args/1]

+			repend boot-code either word? args/1 [['change-id/user/group 
mold args/1]] [['change-id/user args/1]]
 		]
 		group: [word! | integer!] in globals do [
-			repend boot-code ['change-id/group to-lit-word args/1]
+			unless empty? boot-code [change boot-code [change-id/user]]

+			insert boot-code reduce ['change-gid either word? args/1 [mold 
args/1][args/1]]
 		]
 	]
 ]
\ No newline at end of file

The biggest problem was that it was switching the user and then the 
group - but after switching the user away from root, you have just 
given away your right to switch your group. So this never worked, 
and with the processes still having the group root, they could still 
access almost everything on a system, so there was hardly improved 
security

This is hard to see in the standard process list on Linux, because 
you have to use ps u -G <group>. These errors are also not logged

The new code reverses this order to make it work

IDs that you specified were looked up in the user file - also if 
you specified the group. Although in most cases on most systems, 
user names have a matching group name with the same number, this 
is clearly wrong

Specifying just the user already looked up the group number of the 
user, but this was not used for the group setting. As said above, 
it's not very meaningful to only change the user ID away from root, 
so now specifying only a user changes both the user ID and the group 
ID to the IDs corresponding to the user

I'm not sure how meaningful it is to also specify a separate group 
(different from the group of the user), but if you do it should be 
looked up in the group file instead of the users file, so now it 
does that

The users and groups files are line oriented, but they were searched 
as a whole with PARSE. This can easily go wrong, for example if the 
name appears elsewhere in the file, for example in the comment field. 
They're now searched line by line

I'm not sure if capitalised user names are allowed on Unix (traditionally 
not), but they would be case sensitive, so I also made this search 
case sensitive

The format for the configuration file allows numbers for the specified 
user and group IDs, and this is indeed useful, but they weren't supported 
in the processing functions. They are now

If you specify a number for the user ID, it is not used for setting 
the group, because it is not known if those numbers correspond

In general, the contained functions are quite a bit more flexible 
now

Well, I never thought I would be hacking a web server, so this is 
pretty cool, and Cheyenne makes it very easy :-)

Kaj, thanks for this very usefull work, I'll review your patch tonight.

Great

Seems you've solved a lot of issues there.

Yeah, it was worth spending some time on it

Sure, that's a significant improvement for Cheyenne.

Mod-userdir will be finally usable!

:-)

I think there's only one issue left, for which I don't see an easy 
solution. If you specify both a user and a separate group, it must 
be in that order in the configuration file, because the group definition 
modifies the boot code that the user definition generates. Reversing 
them gives you the group belonging to the user, instead of the separate 
group

Oh, we can have the user definiton check for exisiting boot code, 
of course

Or is the configuration always processed in the order of the module? 
In that case, there's no problem

websockets coming to firefox in next release
http://ajaxian.com/archives/firefox-4-html5-and-native-json-store

After some serious benchmarking with Redis using a couple of php 
libraries, it's become clear that a Cheyenne + Rebol blocks is by 
far the fastest solution.

I think the main issue with Redis is the I/O between PHP and Redis 
itself.. the only way to access Redis data is via a port.


ie: Apache <-> PHP <-> Redis  .. each I/0 adds precious milliseconds

Cheyenne has an advantage, as Rebol blocks are native.


I must admit, I didn't expect these results. Accesss to data stored 
within Rebol blocks  is crazy fast... and very flexible.

I have  a rebol block with 10 million integers.. i can crawl the 
entire block with a foreach [a b] loop in 0.17 seconds

Now if i can just figure out how to reduce the large block into a 
few smaller blocks, and run parallel foreach loops, it will be a 
killer app.

Terry, if you are able to simulate thousands of users, that would 
be a killer app too.

Well, as Doc pointed out, 1000s of simultaneous users makes for a 
VERY popular website.. take the millions  of $ and work it out as 
necessary.

I'm looking for say 100 users able to work on a very large datasets 
quickly.

Doc, would it be safe to say processing large blocks are non-blocking 
with Cheyenne?

What do you mean by that? Cheyenne's UniServe task masters run in 
separate processes, so when one request takes time other task masters 
handle other requests, but within one request your code is executed 
serially, the way you program it. REBOL is single-tasking, after 
all