r3wp [groups: 83 posts: 189283]
  • Home
  • Script library
  • AltME Archive
  • Mailing list
  • Articles Index
  • Site search
 

World: r3wp

[!Cheyenne] Discussions about the Cheyenne Web Server

Kaj
9-May-2010
[8220x3]
At least it can be tested with X11 running. Then I find that Cheyenne 
searches for the GNU C library in several places, but not in the 
place where Syllable Server has it. Here's a patch to misc/unix.r 
that fixes this for both Syllable and GoboLinux (the new version 
that's currently in development):
--- unix.r.original	2010-05-09 03:22:33.000000000 +0200
+++ unix.r	2010-05-10 00:36:14.000000000 +0200
@@ -5,6 +5,8 @@
 		exists? libc: %libc.so.6
 		exists? libc: %/lib32/libc.so.6
 		exists? libc: %/lib/libc.so.6

+		exists? libc: %/System/Index/lib/libc.so.6  ; GoboLinux package

+		exists? libc: %/system/index/framework/libraries/libc.so.6  ; 
Syllable
 		exists? libc: %/lib/libc.so.5
 	]
 	libc: load/library libc
After that I found several more problems that make user switching 
unusable. Eventually I concluded that a rewrite of mod-userdir is 
necessary. I'm working on that and will post a patch after testing
Dockimbel
10-May-2010
[8223]
Kaj: thanks, this mode has never been deeply tested. I'm applying 
your patch in svn right now.
Robert
10-May-2010
[8224]
Doc, no I'm using cheyenne0920 version (maybe quite old but it works).
Dockimbel
10-May-2010
[8225x2]
Cheyenne binaries are encapped with [secure none] header to avoid 
such issues. I wonder if the LAUNCH native used to fork worker processes 
is correctly transmitting boot flags to child processes.
You should try encapping latest version from SVN with your SDK. Let 
me know if it fixes or not your issue.
Kaj
10-May-2010
[8227x14]
Here's my patch to mods/mod-userdir.r:
--- mod-userdir.r.original	2010-05-09 19:28:10.000000000 +0200
+++ mod-userdir.r	2010-05-11 00:45:24.000000000 +0200
@@ -12,40 +12,81 @@
 	on-started: does [do boot-code]
 	on-reload:  does [clear boot-code]
 	
-	get-ugid: func [name [string!] /local file uid gid][
-		if none? attempt [file: read %/etc/passwd][
+	get-ugid: func [name [string!] /local file line uid gid][
+		unless attempt [file: read/lines %/etc/passwd][
 			log/error "accessing /etc/passwd failed"
 			return none
 		]
-		unless parse/all file [
-			thru name 2 [thru col]
-			copy uid to col skip
-			copy gid to col
-			to end
-		][
-			log/error "reading /etc/passwd failed"
+		foreach line file [
+			if all [line: find/case/match line name  col = first line][
+				return either parse/all next line [
+					thru col
+					copy uid to col skip
+					copy gid to col
+					to end
+				][
+					reduce [to-integer uid to-integer gid]
+				][
+					log/error "invalid format reading /etc/passwd !"
+					none
+				]
+			]
+		]
+		log/error "user not found in /etc/passwd"
+		none
+	]
+	
+	get-gid: func [name [string!] /local file line gid][
+		unless attempt [file: read/lines %/etc/group][
+			log/error "accessing /etc/group failed"
 			return none
 		]
-		reduce [to-integer uid to-integer gid]
+		foreach line file [
+			if all [line: find/case/match line name  col = first line][
+				return either parse/all next line [
+					thru col
+					copy gid to col
+					to end
+				][
+					to-integer gid
+				][
+					log/error "invalid format reading /etc/group !"
+					none
+				]
+			]
+		]
+		log/error "group not found in /etc/group"
+		none
 	]
 	
-	change-id: func [id [word! integer!] /user /group][
-		if word? id [
-			if none? id: get-ugid mold id [return none]
-			id: pick id to-logic user

+	change-id: func [id [string! integer!] /user /group /local gid][
+		either string? id [
+			unless id: get-ugid id [return none]
+			set [id gid] id
+		][
+			gid: id
 		]
-		either user [
+		if group [setgid gid]
+		if user [
 			;logger/file.log: join logger/file ["-" id %.log]
 			setuid id
-		][setgid id]
+		]
+	]
+	
+	change-gid: func [id [string! integer!]][
+		if string? id [
+			unless id: get-gid id [return none]
+		]
+		setgid id
 	]
 	
 	words: [
 		user: [word! | integer!] in globals do [
-			repend boot-code ['change-id/user to-lit-word args/1]

+			repend boot-code either word? args/1 [['change-id/user/group 
mold args/1]] [['change-id/user args/1]]
 		]
 		group: [word! | integer!] in globals do [
-			repend boot-code ['change-id/group to-lit-word args/1]
+			unless empty? boot-code [change boot-code [change-id/user]]

+			insert boot-code reduce ['change-gid either word? args/1 [mold 
args/1][args/1]]
 		]
 	]
 ]
\ No newline at end of file
The biggest problem was that it was switching the user and then the 
group - but after switching the user away from root, you have just 
given away your right to switch your group. So this never worked, 
and with the processes still having the group root, they could still 
access almost everything on a system, so there was hardly improved 
security
This is hard to see in the standard process list on Linux, because 
you have to use ps u -G <group>. These errors are also not logged
The new code reverses this order to make it work
IDs that you specified were looked up in the user file - also if 
you specified the group. Although in most cases on most systems, 
user names have a matching group name with the same number, this 
is clearly wrong
Specifying just the user already looked up the group number of the 
user, but this was not used for the group setting. As said above, 
it's not very meaningful to only change the user ID away from root, 
so now specifying only a user changes both the user ID and the group 
ID to the IDs corresponding to the user
I'm not sure how meaningful it is to also specify a separate group 
(different from the group of the user), but if you do it should be 
looked up in the group file instead of the users file, so now it 
does that
The users and groups files are line oriented, but they were searched 
as a whole with PARSE. This can easily go wrong, for example if the 
name appears elsewhere in the file, for example in the comment field. 
They're now searched line by line
I'm not sure if capitalised user names are allowed on Unix (traditionally 
not), but they would be case sensitive, so I also made this search 
case sensitive
The format for the configuration file allows numbers for the specified 
user and group IDs, and this is indeed useful, but they weren't supported 
in the processing functions. They are now
If you specify a number for the user ID, it is not used for setting 
the group, because it is not known if those numbers correspond
In general, the contained functions are quite a bit more flexible 
now
Well, I never thought I would be hacking a web server, so this is 
pretty cool, and Cheyenne makes it very easy :-)
Dockimbel
11-May-2010
[8241]
Kaj, thanks for this very usefull work, I'll review your patch tonight.
Kaj
11-May-2010
[8242]
Great
Dockimbel
11-May-2010
[8243]
Seems you've solved a lot of issues there.
Kaj
11-May-2010
[8244]
Yeah, it was worth spending some time on it
Dockimbel
11-May-2010
[8245x2]
Sure, that's a significant improvement for Cheyenne.
Mod-userdir will be finally usable!
Kaj
11-May-2010
[8247x4]
:-)
I think there's only one issue left, for which I don't see an easy 
solution. If you specify both a user and a separate group, it must 
be in that order in the configuration file, because the group definition 
modifies the boot code that the user definition generates. Reversing 
them gives you the group belonging to the user, instead of the separate 
group
Oh, we can have the user definiton check for exisiting boot code, 
of course
Or is the configuration always processed in the order of the module? 
In that case, there's no problem
Terry
11-May-2010
[8251]
websockets coming to firefox in next release
http://ajaxian.com/archives/firefox-4-html5-and-native-json-store
Terry
14-May-2010
[8252x2]
After some serious benchmarking with Redis using a couple of php 
libraries, it's become clear that a Cheyenne + Rebol blocks is by 
far the fastest solution.

I think the main issue with Redis is the I/O between PHP and Redis 
itself.. the only way to access Redis data is via a port.


ie: Apache <-> PHP <-> Redis  .. each I/0 adds precious milliseconds

Cheyenne has an advantage, as Rebol blocks are native.


I must admit, I didn't expect these results. Accesss to data stored 
within Rebol blocks  is crazy fast... and very flexible.

I have  a rebol block with 10 million integers.. i can crawl the 
entire block with a foreach [a b] loop in 0.17 seconds
Now if i can just figure out how to reduce the large block into a 
few smaller blocks, and run parallel foreach loops, it will be a 
killer app.
Henrik
14-May-2010
[8254]
Terry, if you are able to simulate thousands of users, that would 
be a killer app too.
Terry
14-May-2010
[8255x2]
Well, as Doc pointed out, 1000s of simultaneous users makes for a 
VERY popular website.. take the millions  of $ and work it out as 
necessary.

I'm looking for say 100 users able to work on a very large datasets 
quickly.
Doc, would it be safe to say processing large blocks are non-blocking 
with Cheyenne?
Kaj
15-May-2010
[8257]
What do you mean by that? Cheyenne's UniServe task masters run in 
separate processes, so when one request takes time other task masters 
handle other requests, but within one request your code is executed 
serially, the way you program it. REBOL is single-tasking, after 
all
Dockimbel
15-May-2010
[8258x2]
Terry: you need to be more specific, "processing" is very vague. 
Also are you talking about running code in CGI/RSP or in Cheyenne's 
main process?
Kaj: I keep having big issues starting Cheyenne as not-root user, 
the debug files are created while Cheyenne is stil root...The issue 
is deeper. The UNIX port number <1024 limitation, is a true PIA. 
Does someone know if there's a way for a process not started as root, 
to temporary get root privileges for opening a network socket?
Gabriele
16-May-2010
[8260]
Doc, the way that normally works (eg. apache) is that you start as 
root, open the socket, then drop privileges.
Kaj
16-May-2010
[8261x3]
Yes. However, I had no further problems, because the main Cheyenne 
process keeps running as root, and can thus answer port 80. Are you 
doing it differently, Doc?
Well, that is to say, I couldn't test it to the end, because I currently 
have to use View for the library interface, and my graphical Linux 
doesn't fully support X under non-root yet, so maybe I hadn't advanced 
as far as your problem
I assumed it was just my Linux not being fully configured yet
Graham
17-May-2010
[8264x2]
Is there a core version of cheyenne for windows that can run as a 
service without issues?
Has anyone created any multi-tenanted applications?
Terry
30-May-2010
[8266]
what would cause sudden 403 Forbidden errors on certain pages? Everything 
was fine, restarted the server.. 403
Oldes
30-May-2010
[8267x2]
403 is also Access Restricted, so it depends what's in your settings.
But as I'm checking the sources, in Cheyenne itself there is no code 
forcing 403.
Terry
31-May-2010
[8269]
nvm - i had a rogue apache service firing up on boot.