AltME groups: search

on mac, the my copy of the repo had a custom httpd.cfg, which i moved 
away, but now it doesn't want to regenerate it

let me try a clean checkout

Looks like a user permisson issue?

svn can only assign exec flags in a post-hook, right?

is this hash bang thing under windows also a cheyenne feature?

Ok, let's have a look at your log files proposition...

Doc, thanks... had only the service/process-queue... added the process-next-job() 
function and calls... 

seems like I'm getting an error elsewhere now (later in the reply 
chain), so that's a good sign.  thanks.

it would be nice to have a little options in the cfg file... something 
like

-trace-log %/path/to/folder/

well, it took me a month to realize where it was.... in my REBOL.exe 
folder.   :-)


the problem with using "current-dir" is that it can be anywhere, 
especially on windows where the concept of a current-dir is pretty 
clueless.

just for example, depending on how you start a rebol application 
(shell, drop icon, link), there can be 3 different current dirs...

You can change that easily if you're running it from a shortcut icon. 
Edit shortcut properties and set the "working folder" where you like 
it to be.

yeah, but I want that kind of thing to live within the configs... 
 since... its a configuration  ;-)

anyhow.. I can hack it in quite easily... its not that big a deal.. 
just a suggestion  :-)

I won't make a per log file configuration option, that wouldn't make 
much sense. When I'll had such option, it will redirect all log files 
to the same folder. Still need to finish reading onetom proposition 
to see if there's a better solution.

it should also go to a %log/ folder

meanwhile i wrote a little script to fixup the cgi hash-bang lines 
and give exec flag on them

Doc,    wrt handler.. it really was a stupid error on my part... 
I didn't set the   req/out/code   so I guess it stays in limbo in 
its worker handling, since its not valid for request handling, but 
it has no return error code for reply handling either...

If I have a single wish, it would be for all of the mods and handler 
scripts to have a verbose rebol header.  it would help a lot understanding 
the hierarchy and purpose of some of the mods.

I agree about the headers: REBOL [ ] is a bit short. :-)

Maxim: please throw together a blog entry at least about your rambles, 
so later we can use it to extend the documentation

Running Cheyenne with -vvvvv (should be enough) will give you a view 
of the matrix (line by line, per event) when Cheyenne starts.

Dockimbel: I have to thank you for your presentation. Just saw it 
yesterday finally. Now I understand Cheyenne a bit more. Im curious 
about the tricky parts though. Like how do u handle forking and the 
communication between the master and the forks and stuff like that. 
I know it's handled by UniServe

Forking is done when Cheyenne starts, more precisely when the task-master 
service is loaded by UniServe. It starts with a default of 4 workers 
processes. If more than 4 simultaneous RSP or CGI requests are received, 
one or several workers are forked (up to 8). You can change those 
values in %cheyenne.r (search for pool-start and pool-max). It is 
planned to expose those values in the config file for easier access.

Oh, well, you got a quick explanation right now. :-)

Doc yeah... I'm starting to be pretty knowledged on the phases (though 
some things still elude me a bit).


I'm just saying that some mods... like extapp, expire, action, alias, 
internal, etc...  probably have non-negligible effects in the chain 
that some other modules might react to or not (depending).  and its 
currently quite tricky to understand what effects the mods actually 
have on requests and response.


anyhow, as usual, Its not critique its just observation from someone 
"outside the box"   :-)

Communications with the workers are handled by the task-master UniServe 
service. It is done with a TCP channel (Cheyenne/UniServe process 
is the server, workers act as clients). When a worker is launched, 
the first thing it does is to connect to the server and wait for 
a new job. If the TCP connection is lost, the server considers the 
worker dead and forks a new one to replace it. Workers are doing 
their best to maintain this (umbilical) link open, if they encounter 
a fatal error, they severe the link before quitting.

Max:"its currently quite tricky to understand what effects the mods 
actually have on requests and response"


Agreed, some mods, like mod-static or mod-action are catching a lot 
of events and doing important things. The only way to not get lost 
is to treat it horizontally (from the slide), it means see list all 
mods catching a given event, see the process order and the exiting 
condition: 
- none: not processed

- false: processed but let other lower priority mods callback catch 
it too

- true: processed and stopping the event propagation to lower priority 
mods callback

btw, these lots of ports might look scary for the usual paranoid 
sysadmin, so it's not a good selling point. by default these all 
should be closed, but only port 80 or 8080 or something like that 
should be open

Dockimbel: you r actually writing an awesome documentation rightnow. 
if i can get access to the wiki, i would quote these on a sandbox 
page, so later we can massage it into an actual documentation

onetom: those ports I listed are for Cheyenne internal usage, they 
should never be reachable from outside on a properly configured server 
(requires a firewall as REBOL binds ports to all interfaces). You 
should just let the HTTPd ports be reachable from outside.

onetom: if you need a wiki account, let me know.

I've been using Cheyenne for personal web server for a few years 
now ... only issue was when user installed it as a service

usual paranoid sysadmin

: If I was a fulltime sysadmin, I would be much more scared to put 
any PHP app online than to run a webserver written in an obscure 
language. ;-)

Could be a nice selling point.

(do u seem my private msgs? im not sure what should i see / notice 
when i get a priv msg in altme...)

onetom: I started writing you a reply by email, but it might be easier 
to discuss it there.

onetom... ever since the weird user reset bug which affected altme, 
the very first time anyone ever sends you a personal message, you 
won't be notified... you actually have to click on the group to see 
the message updated  :-(

let's collaborate on a directory structure here: http://piratepad.net/KkvkZ9AtME

Those "private" folders are protected from the requests by the webserver, 
but not by the OS. If an attacker finds a way to workaround the server's 
protection, better avoid giving him critical info.

where they saw a lot more than their interest

running their own app server (mongrel/thin/unicorn) helped a littlebit, 
but they couldn't easily test what was returned just by the webserver 
from cache / static files

i had to edit config files when a new user joined the company

security in such a case is not a big concern (unless the machine 
get rooted :)

Cheyenne just needs to be able to pick the config file from a remote 
folder instead of just current one.

sure, but i would love to get rid of any external component, since 
cheyenne seems to be pretty close being able to cater for such a 
setup

if it still looks in the current directory for a httpd.cfg by default, 
then it's not even necessary

not a big difference

why not a mod_log  ;-)

any relative path in a config file is normally calculated from there

this mod-userdir name is a bit misleading, i think. i was expecting 
some handler for the /~user/  pathes and the ability to automatically 
fire up workers with the right ownership based on the owner of the 
directory it's serving from

i just need the logs go out of the way in a binary version

i was thinking making a webapp which returns on-page-end the to-json 
context load rejoin [%. request/parsed/file %.r]

but i get a 404 in an earlier step, i guess

i have a couple of ideas how to hack this, but what would be the 
"correct" way?

hmm.. now how can i write a module for the binary version of cheyenne? 
can i?

I made a start on a mod-rest but can't access it at present .. looks 
like the server has changed ip address behind nat

ok, now, i have a more interesting problem

i value the intent of handling the variables in these 2 places in 
a uniform way, but concatenating them at the raw level doesn't make 
much sense

and this is the http request:  curl  -d '{"a": 1, "b": 2}' http://localhost:8080/?test

i wouldn't be happy to start a separate webserver just for a dumb 
rebol-json gateway

json.rsp: <% print to-string request/content %>


>> probe read/custom http://localhost/json.rsp[POST {{"a": 1, "b": 
2}}]
connecting to: localhost
{{"a": 1, "b": 2}}

>> probe read/custom http://localhost/json.rsp?test[POST {{"a": 
1, "b": 2}}]
connecting to: localhost
{{"a": 1, "b": 2}test}

Ok, I see the issue now.

>> system/version
== 2.7.8.2.5
>> body-of context [a: 1 b: 2]
== [a: 1 b: 2]

if there is such a thing :)

I guess my Cheyenne build scripts are missing a R2/Forward specific 
include.

json.rsp:
<%= probe request/content %>
<%= probe as-string request/parsed/arg %>
<%= probe as-string request/parsed/content%>
<%= probe request/parsed/arg %>


>> probe read/custom http://localhost/json.rsp?test[POST {{"a": 
1, "b": 2}}]
connecting to: localhost
{[{"a": 1, "b": 2} "" test ""]{"a": 1, "b": 2}test
{{"a": 1, "b": 2}}{"a": 1, "b": 2}
{{"a": 1, "b": 2}}{"a": 1, "b": 2}
test
test
}


Some corruption is occurring, request/parsed/* should be left untouched.

thx, doc for the quick responses! i feel a bit quilty though, coz 
u can't work on red instead, but hey, let's make what we already 
have bugfree 1st. these needs to be fixed even if we run it over 
red, right? :)

Ok, false alarm, no bug in RSP request decoder:

json.r
<%= mold request/content %>
<%= mold as-string request/posted %>
<%= mold as-string request/parsed/content %>
<%= mold request/parsed/arg %>


>> print read/custom http://localhost/json.rsp?test[POST {{"a": 
1, "b": 2}}] none
connecting to: localhost
[{"a": 1, "b": 2} "" test ""]
{{"a": 1, "b": 2}}
{{"a": 1, "b": 2}}
test

The <%= in addition to PROBE was producing a misleading output.

i have a couple of ideas how to hack this, but what would be the 
correct" way?"


You could try using the ALIAS config option. Add this to the httpd.cfg 
file in a domain section (not sure if it would work in a webapp context):

    alias "/rest" %rest.rsp

and call your REST resources with:
    /rest/asd 
    /rest/qwe


In %rest.rsp, you can put your URL parsing code to produce a "REST 
routing" and return the JSON object. It might also work with "/" 
(untested).

Doc, wrt req/in: make req/in   []    so far it works flawlessly. 
 I'm using this to store computed data from an early phase and sending 
it to the handler later.  This allows me to prevent double or tripple 
processing the same thing (a decision based on the url)

Extending the req/in object! is a good solution. I should maybe add 
a 'user field in request object for storing custom mod data...

don't know, since then more than one mod might start to clobber it, 
and we'd be fighting for the space (same difference, just one level 
down  ;-)


and, since those extra bytes get molded/sent/loaded to and from handers... 
it seems like waste when for most people this data will not be used. 
   I'd actually rather document how a module should expand a request... 


like spelling any new word, starting with the mod's name to prevent 
collisions.

I tend to use COLLECT a lot in REBOL scripts, but it seems it can't 
be used with Cheyenne.

QUERY is a REBOL native function

The issue with COLLECT is caused by DO. In RSP environment, DO is 
redefined to bind argument code to webapps execution contexts. The 
 RSP mezz DO has a fixed arity (= 1) while the native DO has a variable 
arity (= 3 inside COLLECT), hence the error.

Ok, I've found an (obvious) patch for COLLECT. A reference to the 
DO native is kept in RSP (*do). So the following patch (run in global 
context) would solve the issue:

if pos: find second :collect 'do [pos/1: '*do]

ChristianE: I've commited a fix to Cheyenne public repository. It 
will be available as a new source package in http://cheyenne-server.org/download.shtml
in about an hour. But I won't build new binaries now (takes too much 
time).

If you're using the binary version and don't have the SDK, you can 
put the one-line patch in a REBOL script and load it in your app's 
startup code or better, load it using the 'worker-libs config keyword.

Nenad, that's great! Thanks for all the work and effort you put into 
Cheyenne! And don't even think of taking any extra time to build 
binaries for just a little fix like this one, really, I can wait 
and happily use the patch.

Doc, one thing I have not yet fully mapped out in my mind wrt handlers.

Q1:

how do the handlers actually compile/get their source code... is 
it sent over tcp and run there, or does the handler load a file on 
its own?

Q2:
when exactly does this happen?

Q3:

can I configure the handler source or data in some way before it 
gets compiled/executed,  (at the mod conf parsing stage).


I neet the handler to share some data with the mod which manages 
it in some way.


I don't want to send this config data via the request, at each request 
(makes no sense)

ah... don't bother, I just figured out... I've been reverse engineering 
the whole task-handler/master code for a few hours... finally found 
the magic lines I was looking for.  


its neet, you load the files on demand, loading each module, only 
if its actually required.  best of both worlds.

so I guess, the best way to configure the handlers through cheyenne's 
httpd.cfg system,  is to save out a temporary configuration file 
on the fly when the conf parser is run.  


this file would be executed by my handler when it is launched by 
any of the worker processes.

yeah... I've changed my solution a bit.   the handler now has a variable 
to know if its been configured yet, and if it has, it skips that 
part of the code.   so even if I get the cfg at each request, I ignore 
it.


The cool thing is that my mod really doesn't need much configuration 
and most other standard configs are useless to it, so I commented 
just about all of them in my httpd.cfg and my mod still works.  :-)


very lean on the cheyenne side.  I do all the heavy lifting in the 
handler, so it can scale pretty well.

maybe I could add a little cfg option for them... my client is going 
to need this.

btw, I'm starting to feel like a cheyenne mod/handler code ninja 
 ;-)

You can, except you, I think only Philippe Legoff got that far :-) 
BTW, he wrote a few nice articles (french only) on Cheyenne usage: 
http://pl.legoff.free.fr/dotclear/rebol-fondation/index.php/?q=cheyenne

I'm having a strange problem with one virtual-root that works and 
another that doesn't. The one that doesn't simply sends back a 404.

What's the best way to debug app-init.r now? I can't see whether 
a bug happens or whether it runs ON-APPLICATION-START at all.

i have an rsp like this:

<% probe delta-time [ ... switch request/method [ get  [...]   post 
[...] ]  ... ] %>

if i make a GET, it's fast:

$ time curl -s -D- http://localhost:8080/docs/rfq3 >/dev/null
0:00:00.003815
real    0m0.026s


if i make a post, the rsp part is still fast, but the overall request 
is damn slow (even consequent requests too):

$ time curl -d @xxx -D- http://localhost:8080/docs/rfq3
0:00:00.004595
real    0m2.034s

it shoudn't send it.  the servers are always free to implement more 
headers than the spec.

also realize that if you use http 1.0 there are no virtual hosts!! 
supported so its a bit deal.

If you haven't figured out the COLLECT rewrite yet, let me take a 
look.

Here is a workaround (by raping curl):


$ time curl -H 'Expect:' -d @xxx -D- http://localhost:8080/docs/rfq3
real    0m0.082s

HTTP code 100: not supported, if you have a (not obscure) use-case 
for that, I might add support for it.

BrianH: I've found a workaround for COLLECT (just replacing DO by 
*DO which refers to the native function in RSP environment). Can 
you tell me if there are other recently added mezzs (from R2/Forward 
mainly) that rely on DO in their implemenation?

(should be a one-line patch like for COLLECT, so not a big deal)

Both of them do a block though.

Oh, if it's a block!, then it should work in RSP.

IN-DIR also does a block.

In APPLY, the QUOTE function is added to the constructed block containing 
the function call. If you don't do something fancy with that function, 
no rebinding is needed. That function should really be protected 
in the runtime though; a *lot* of runtime code absolutely depends 
on these low-level control functions to behave *exactly* as they 
are expected to. Same goes for ALSO.

personally, i never had to use such negotiations which wouldn't accept 
a request..