AltME groups: search

thanks... wrt CRON... when you release the next version, I could 
drop that into it and send it back to you.  you don`t have to do 
everything yourself   :-D

I'd like to not do everything by myself, but it's not that easy. 
I have some deep concerns for Cheyenne core part such as speed, memory 
usage, stability and security. Cheyenne has become a *critical* part 
of our business, I have to garantee that a new version won't break 
our webapps in production, nor make them instable, insecure or noticeably 
slower. My responsibility also extends to other companies that are 
selling products or services based on Cheyenne.


I've already accepted small patches on Cheyenne core in the past, 
but it takes me a lot of time to study and test each line of code 
an rewrite them if required. If your code has only a local impact, 
I might use it, if it needs to patch a lot of parts of Cheyenne/UniServe, 
I probably won't. Anyway, you can send it to me, it's always a good 
inspiration to see how other developers solved some specific problem.

I can't disclose much right now, one of our Cheyenne based product 
is getting a lot of attention recently and in a couple of weeks, 
we will know if we need to hire a few more peoples at Softinnov. 
;-)

A few small intranet apps in a couple of TOP 5 french company are 
using Cheyenne and RSP.

And we have also a big product using Cheyenne/RSP with already a 
few dozens customers.

Dockimbel, let's say R3 was done and most bugs were squashed, would 
you then build Cheyenne for R3 and would it be from scratch?

I don't see it anymore because now I use a Rebol client to access 
Cheyenne and not a web browser.

R3: when it will be feature complete and in final beta stage, sure 
I will. I'll probably rewrite complety the lower level networking 
code and try to keep as much as possible the higher level code.

what role would uniserve play, if networking is completely async 
and threading is possible in R3?

graham, you can run MANY cheyenne servers on the same system .  and 
they can be handling several thousand requests / hour each without 
failure.  


at my client cheyenne is probably the most stable server application 
they have, a part from apache.

UniServe still has a purpose in R3, but it implementation will be 
much lighter and it will run much faster. Btw, one of UniServe's 
plugin, Task-master, is in charge of running and exchanging data 
with external processes given true multitasking abilities to UniServe's 
based products (RSP scripts are evaluated in such helper processes). 
R3 multithreading will make multitasking much simpler and way much 
faster.

the mod for access refusal is finished btw.  it works really well, 
I ended up doing it in a mod and doing a few invisible actions within 
the make-response and task-done callbacks.

and obviously, you musn't have two servers with the same ports.

as Henrik said.. cheyenne was certanly rebol "web-window" for me. 
The day I tested and saw it can handle 300req/sec I switched to rebol 
for webdev.. there is no way I would use ordinary CGI to make webapps 
at this time.

Well, maybe you can't. I haven't given any thought to what it takes. 
I was only thinking of the basics like large file transfer and proper 
working async ports and threading. some basics that a good webserver 
can do.

right now, I can tell you that cheyenne, from the client's point 
of view, does all of that.  R3 will just allow it to be a bit faster, 
probably a bit more robust at the seams, and definitely easier to 
support, since some of the workarounds will now be implemented directly, 
and whatever is missing, doc can add/fix directly in binary.

libevent was suggested in the past, along with links to liboop etc. 
Not sure the licence is OK. Anyway - I wonder where do we go such 
a way? I can already imagine complete mess and tens of versions of 
custom R3s, if such low level things as main event loop are open-sourced 
and replaceable.

pekr, who cares if there are 2000 versions of compiled rebol out 
there.   RT's is always going to be the default, and all the rest 
will be purpose built.  at least now we can play with any other tool.

for example, i know of a server which profiled the tcp stack of their 
server and realised that some buffer sizes didn't get cached the 
same way through windows.

just changing the size of buffers and tcp payloads added a big speed 
boost.  stupid detail, but now if we have such cases, we can actually 
really go as deep as that.

BrianH: how do you want to bring something like Cheyenne (Uniserve) 
to R3, if such low level stuff as concurency is not designed yet? 
Wouldn't it (using tasks or not) influence its design? So do we wait 
for new R3 concurency model, or do we proceed with protocols, and 
rewrite later? (we can move to R3 chat instead)

BrianH: I think that what Doc means by Windows console is R2 kind 
of console, not that ugly black monster Windows offers you by default 
:-) There was lots of talk about the console topic and we imo need 
both - system default one, for admins, and GUI based one, for normal 
users. But GUI console could be created using View, as Cyphre showed, 
even in R2 it was nicely usable ...

Yeah, the intention is that a GUI console will be written in REBOL, 
part of the community-created, open-source portion. Then you can 
use or adapt the console for your own apps as well if you like. How 
about having RConsole being implemented with that? :)


Right now the GUI doesn't have good-enough Unicode support to make 
the console yet, so the GUI console will have to wait for the release 
of the host code (the current priority), and the subsequent resumption 
of the GUI work.

Be sure to select the QuickEdit Mode and Insert Mode options for 
the DOS console - they make your life easier :)

I think I have found a bug where Cheyenne keeps serving an empty 
file, if I have first had it put in a server directory as a MacOSX 
shortcut and then replaced it with a real file of the same name.

is there a way to prevent caching and logging?

logging: See %changelog.txt (search for disable-log and no-log)

If you want mod-remark to take other mod-static caching, just provide 
a 'make-response handler in mod-remark, give it a higher priority 
and make sure to return a TRUE value (that will end the phase shadowing 
mod-static's handler for that phase). You can also just unload mod-static 
by commenting it inside GLOBALS section in config file (you'll have 
then to provide all the adequate handlers for serving static resources).

the thing is that in order to save resources, mod-remark will be 
creating MANY static files (including images and css scripts) but 
the content of those pages can ultimately change at each refresh, 
if the user is editing preference type information.

Just remember one important thing : mods live in the main process 
(the one running UniServe), so you have to keep mod handlers fast 
enough to not slow down the whole server. That's why Task-master 
service and it's helper process are here, to handle the load for 
the main process. See mod-action as an example of unloading the main 
process from the burden of running CGI scripts.

yep.  I know all about the task handlers... they are one reason I 
was having difficulty tracking where and how dynamic code was being 
run in my client's app.   ;-)

Looks like you've gone really deep in UniServe. Sending job to helper 
processes has a cost (building message, MOLDing, transmitting through 
TCP, LOADing, decoding message). With R3 and multithreading, this 
cost will reduce to zero, thanks to shared memory (or equivalent 
system).

I use code like this:

<input type="radio" name="paymethod" value="paypal" />
<input type="radio" name="paymethod" value="somethingelse" />

And this code is wrapped in a DIV.

So, it looks like everything from client to server, to cheyenne is 
passed correctly. And than inside the RSP page it's lost.

No, I load the answer HTML page and insert some HTML on the fly before 
returning the page via a PRINT.

how do I use rconsole to tell cheyenne to reload a mod that has changed 
on disk... 


or does cheyenne detect mod-file changes and reload them automatically?

You can't. Mods are part of Cheyenne's kernel and the priority of 
each mod's callback is determined relatively to the other mods during 
Cheyenne boot (kind of competition for higher priority for each phase). 
Reloading a mod would required reloading all the mods, breaking most 
of the active connections (CGI, RSP, FastCGI,...). So the answer 
is : kill and reload Cheyenne.

with your permission I would like to rebrand ssh-admin to  Cheyenne-admin.

its like a stand-alone cpanel for cheyenne-based systems.


All the current file and system commands options to manage the host 
server too will be complimented with any tools needed to configure 
and control cheyenne and any mods which are installed.

hehehe, will be providing the link to my site once its on-line, and 
then will start working on revault, for which I also purchased domains.

already using cheyenne-admin to setup my bash user config files (~/.xxxx 
files) copying files locally, editing in ultra-edit, and uploading 
them back  
all via one click for xfer   :-)
I'm lazy  hehehehe

what we need with !cheyenne are minial examples of services and mods 
which implement all callbacks and give just a small comment on why 
and when it is called... this would help soo much in understanding 
how to implement cheyenne extensions.


I am currently looking at the rsp code and its so huge and complex 
that its a big daunting to grasp the whole by looking at its parts.

Mod-RSP is the most complex one and it does a lot of things. Sessions 
in Cheyenne are RSP-specific. There's a synchronization system in 
RSP sessions for protection of session data from corruption in concurrent 
executions. That may be easy to understand by just reading the code. 
Anyway, feel free to ask for explanations of specific code parts.

there are too few comments (read as almost none ;-) in your code, 
 and that doesn't slow down server in any way   ;-D

It works using a semaphore (session/busy?: yes|no). It used to be 
very fined-grained with distinction of RSP scripts just reading session 
data from scripts modifying session data, but it was even more complex 
to maintain and the RSP source static analysis I was doing could 
never be 100% accurate (it''s easy to bypass any form of static analysis 
in R2). So now, it just blocks concurrency inside a user session 
(only 1 RSP script per user session allowed).

Session data is passed to task-handlers along with each request and 
saved back in main process on task-handlers response.

It's always possible to write session data to disk or to a DB and 
rely on the DB locking system to ensure that data is safe, but there's 
a performance overhead in that case that I'm afraid, may not result 
in any speed gain for the user. It also requires to have a DB engine 
installed on all servers running Cheyenne using RSP sessions, which 
might be overkill in some situations.

this being said... mod-remark will be using persistent liquid nodes 
for session and post/get parameter side of things, and its possible 
that I might store session data outside of the server itself, using 
a liquid-tcp node. 


so that access to actuall session data could be brokered outside 
the server's jurisdiction, extremely efficiently, locked one value 
at a time. and any access/change could be perpetuated to other handlers 
when they attempt to lock to one session value.

you could actually have one page sending xml requests to the server 
in ajax, and another page refreshes with the results of those requests...


the second page could also send a page request with the some parameters 
sent, and the server will reflect all changes to the current session/page 
so far.

I thought also about doing a TCP based session server (with session 
variable granularity), but the overhead in synchronization would 
hit performances (and scalability) too much IMO. If each time you're 
setting a new value to a session variable in a script, you have to 
query a TCP server (even a local one), that would have a huge impact 
on a script execution time.

might still be much faster than a db call.  but the real game-changing 
aspect of mod-remark is that a document in ram, will only render 
parts of itself which are changed, or which cannot be cached, so, 
unless you've actually changed a parameter, it knows that it hasn't 
changed and will not need to ask for the broker to give it back the 
value.  in fact, it wont even process that part of the page, only 
using the data in its local node cache directly.

a news reader could have all the news items cached already and all 
its doing is assembling those which are matched via the request... 
really only doing a fast rejoin of many items, and then a rejoin 
of header news and footer and then spit it out.

all the code which was originally called to create the news items 
is skipped... until some site parameter changes how the news items 
are built, which will automatically cause the news item to reprocess... 
 but on the page and server side, nothing special needs to be done. 
 liquid's internal caching will fork the processing by itself, and 
then start serving the new cached data the very next time any single 
news item is asked for again.

That's quite close to how RSP are working. Each RSP script is compiled 
to optimized REBOL code and cached in memory. A RSP request will 
then result in evaluating the REBOL code which mostly just append 
static and dynamic parts in the output buffer.

the difference is that with liquid, its explicit, and I can explicitely 
share some data accross many scripts, but just some of them.    the 
headers can be half cached, where each page uses half the common 
header, inserts its local menu into it, and that header is now permanently 
cached for all further requests of that page, but it used part of 
data which was already cached by another page.

query-string: Ah, the problem comes from a wrong rewrite rule for 
GET requests. I need to figure out how to handle this case.


That's what I like about web-stuff: There are so many possibilities 
and places that something can fail...

its actually a global property of php that you can change the url 
parameter separators for your site in order to fight back against 
bots and make hacking a bit more complicated.

so most people put their time on tackling the real problems like 
sql injection and https use properly

and use https properly.

doc, I have noticed a usage in your mods which you might want to 
change for speed reasons.


you use the word return a lot... and in my tests, it causes a BIG 
performance hit on function calling... I really mean noticeable when 
you do loop profiling ... a minimum of 20% slow down IIRC.

so instead of:

    phase: func [svc req conf][
        if declined? [return none]
        ...
        if let-others [return false]
        ...
        true
    ]

you really should be doing 

phase: func [svc req conf][
        if accept? req [
              ...
             true =  let-others? req
       ]
] 

just my two cents.

its just a question of maximising performance, and I know dockimbel 
is very serious about improving cheyenne's .

help! cheyenne has no trace.log file in logs, and I can't get the 
verbosity to work without using command-line args which isn't an 
option in my current setup.

but I finaly conceded victory to cheyenne and managed to setup args 
for my startup...

and exactly why can't you run it in verbose mode?

I was studying the req object and it has a log? parameter in the 
out  subobject.

download and double click.... but  I'm working on mod-remark, so 
its not currently serving web pages...

still at first, i did see the standard cheyenne pages and don't have 
any logs.

RETURN usage: your benchmark doesn't reflect real usage. RETURN cost 
is only significant if you didn't spent much time since you've entered 
in the function. In other terms, if RETURN is at the very beginning 
of the function, it might have a significant (means measurable, doesn't 
imply high) impact on performances, if much code has been processed 
before reaching it, I guess that you won't be able to measure any 
difference in performances.


In Cheyenne's mods, I often use a testing expression at the beginning 
and jumping out if it doesn't match. Let's try to calculate how much 
gain I would get by removing this early RETURN :
    - 500 incoming req/s (extreme load conditions)
    - 10 mods
    - 12 callback / mod (each one having a early RETURN)

    - execution time for testing expression before each RETURN : 0 (will 
    give us the maximum possible final gain)

RETURN evaluation time :  (according to your benchmark)
>> (1.032 - 0.296) / 1E6
== 7.36E-7

# of RETURN evaluated under the testing conditions during 1 sec :
>> 500 * 10 * 12
== 60000

Time spent in 60000 RETURN :
>> 7.36E-7 * 60000
== 4.416E-2			; = 44 ms, roughly 1 / 20 sec


So, under extreme conditions, having a testing condition before RETURN 
taking no time, we can have a maximum gain of 5%.

This translates in real usage in a gain of 0 to 5% depending on server's 
load and test branching conditions performances.


Looking at the testing conditions in current mods, I guess we could 
squizze between 0 and 2% (under extreme load only). I'll try to hunt 
down those early RETURN cases in future versions.


Btw, there's a drawback in not using RETURN, you end up with nesting 
IF/EITHER expressions, which gives you less readable code IMO.

Also, setting conditions and using IF or EITHER expressions has overhead 
too.

Are the paremters in request/content somehow checked after receving 
from the client if they contain malisous Rebol code or anything else? 
Or is everything just plain converted to string! and that's it?

verbose is at vvvvv (working) and pages are being served... yet I 
have no *.log files.

thanks for the log-dir config... should put ALLthe configs in the 
httpd.conf file and gray them, like apache does it... with comments 
on what each config does...

Currently yes. I didn't found any value of having logs both on screen 
and on disk at the same time. But if you can convince me that it 
has a value, I may support it in future.

if you had log option and console option within the default config 
file, (commented out or not) then users choose what they want.

my client uses the console for real-time status checking... using 
remote desktop and just noticing if the client isn't serving stuff 
anymore... but the logs then allow you unravel what led to that problem.

Max: I agree the main issue is not having config options documented. 
About the current logging rules, I've always found that's way handier 
to pass command-line options than having to edit a config file. I'll 
see in the next version how I can improve that. 


Btw, I recommend running Cheyenne as encapped binary on production 
servers, it's simplier to handle (especially on Unix) and more secure 
(you can't corrupt some vital source file).

in the httpd.cfg...

listen [83]


I'm using cheyenne on port 81 since I also have apache on my system 
and it works.

the url will be http://mysite.com:81/index.html 

not using vhost though.

Yes its listening to port 83 and I get the default web page (Cheyenne 
test page for now)

If I URL to the "mysite" dir (www.defaultsite.com/mysite) I  get 
vhost index page...

I have no reference to it...
just localhost
and a bunch added by spybot

and if they are set to local host, then you can even trap them using 
a service expecting known ports.

Amacleod: did you tried Apache and Cheyenne using same server port?

no, doc apache is on 82...cheyenne is on port 81 on the server and 
i tried 83 on y other computer..
iI reaching the default page..just not getting vhosts dir

I have a problem, that after some running time Cheyenne seems to 
get into an unstable state and my REST shopping-cart isn't working 
any longer. I got this error in the trace.log, which seems to be 
Cheyenne internal:


5/6-10:09:48.142823-## Error in [task-handler-40014] : Make object! 
[                                                                
                 
    code: 501                                  
                                                                 
                                      
    type: 'access         
                                                                 
                                                           
    id: 
'not-open                                                        
                                                                 
            
    arg1: "Port"                                    
                                                                 
                                 
    arg2: none                 
                                                                 
                                                      
    arg3: 
none                                                             
                                                                 
          
    near: [parse/all current: fourth entry [          
                                                                 
                               
            any [                
                                                                 
                                                    
            
    end break                                                    
                                                                 
        
                | "#[" copy value to #"]" skip (        
                                                                 
                             
                    append out reform 
[                                                                
                                               
                 
       " prin any [pick cat"                                     
                                                                 
   
                        locale/id? value                     
                                                                 
                        
                        mold value #"]" 
                                                                 
                                             
                   
 ]                                                               
                                                                 
 
                )                                              
                                                                 
                      
                | "<%" [#"=" (append out " 
prin ") | none]                                                  
                                          
                copy value 
[to "%>" | none] 2 skip (                                        
                                                          
      
              if value [repend out [value #" "]]                 
                                                                 
              
                )                                 
                                                                 
                                   
                | s: copy value 
[any [e: "<%" :e break | e: "#[" :e break | skip]] e: (          
                                                     
           
         append out reform [" txt" index? s offset? s e #" "]    
                                                                 
         
                )                                      
                                                                 
                              
            ]                     
                                                                 
                                                   
        ]]   
                                                                 
                                                                 
       
    where: 'confirm                                      
                                                                 
                            
] !                                 
                                                                 
                                                 
5/6-23:01:46.501455-## 
Error in [task-handler-40014] : Make object! [                   
                                                              
  
  code: 501                                                      
                                                                 
                  
    type: 'access                             
                                                                 
                                       
    id: 'not-open        
                                                                 
                                                            
    
arg1: "Port"                                                     
                                                                 
                
    arg2: none                                  
                                                                 
                                     
    arg3: none             
                                                                 
                                                          
    near: 
[unless no-lang [                                                
                                                                 
          
            id: locale/lang                           
                                                                 
                               
            locale/set-default-lang 
                                                                 
                                                 
        ]      
                                                                 
                                                                 
     
        out: make                                          
                                                                 
                          
    ]                                 
                                                                 
                                               
    where: 'confirm 
                                                                 
                                                                 

] !

Any idea what's going on? I could install a CRON job to killall instances 
and restart Cheyenne every 24h but IMO that shouldn't be the case.

I have cheyenne (previous version) running 2 sites with moderate 
load for 2 months on some VPS and I never needed to reset it.. onother 
VPS I have the latest version running for about amonth too

Mostly nothing. Cheyenne is working as a reverse proxy and just servers 
an RSP file.

I just use GET and POST.

It's just a intermediate binary, it's not the full 0.9.20 version 
I'm planning to release, there's still a lot of small fixes and enhancements 
to add.

doc: might I do a RFE (request for enhancement)?


add a ./conf/  dir to cheyenne and load every file that ends with 
.cfg


this would allow us to distribute a configuration file with a module 
and provide setups per mod... its much more flexible to manage.

we could also have a setup for each vhost in our system, if that 
makes sense for the web admin.

cool.  and I'm reiterating the need to provide a sample file ala 
apache with a paragraph of comments or two which explain all configs... 
just in case you forgot to note it... this for me is big hassle.


for example... the subtleties behind 'BIND  and 'BIND-EXTERN  are 
not obvious to deduce just by the name... 
-what is the difference in how they are cached? 

-is an external handler explicitely needed with 'BIND-EXTERN   (no, 
in fact,  but it enables it)
-how does one use an external handler?

yes... I already new... but I had to trace the code and lost some 
time wondering why my page wasn't being re-rendered when I first 
used 'BIND  ;-)

I also had to trace the logic to make sure that cheyenne wasn't actually 
expecting an external handler if I used 'BIND-EXTERN...


I ended up loosing more than an hour to figure it out myself... now 
that is just one config... there are MANY... a lot of them I don't 
even know exist.


the above is exactly the kind of information which should be included 
within the httpd.cfg file, even if an example is commented out, but 
provided as an example use.  just like apache does it.

I am trying to make mod-remark as consistent and integrated as possible 
with the rest of cheyenne.  the end goal being that you remove the 
rsp from your modules and drop-in remark instead.

I've scrapped the previous remark system in favor building remark 
v3 right away.  this will actually help me build the mod much faster 
and will provide 100% dataflow engine from its first release.  every 
single programmable entity within mod-remark is now based on a plug. 
 the architecture I have now is becoming very orthogonal... instead 
of building up different objects for each level of config, I think 
I'll be able to reduce it to ONE.


these models will serve as references for the !compilator to create 
persistent  !documents... note that !documents are multi-leveled... 
you build documents by linking up document together.... so if only 
part of a !document is dynamic, only that part will cause processing... 
and by dynamic, I don't mean that its cgi... I mean it has actually 
changed.  down to a single HTML element.   that's what I am aiming 
in any case.


!documents can be stored at any level... from server down to specific 
page and single session. caching is embeded in liquid so it should 
be pretty fast, and inter document data sharing should allow us to 
make it very RAM efficient too.

I hate Apache config file. Because I hate having to read tons of 
docs to just "switch on" some app. Cheyenne's config file has never 
been designed to copy the Apache way, nor to be used by average end 
user. It's just a placeholder waiting to be replaced by a builtin 
web GUI allowing a simple, fast and straightforward way to manage 
the server. That has been the plan since the beginning and one of 
the main motivation for building Cheyenne. Unfortunately, I never 
had the required time to complete that goal yet, so I'm stuck with 
that, and that's also why Cheyenne is still at 0.9.x.

but the cfg file becomes documentation... that is the real use of 
the text, and it lives directly within system, so you don't have 
to maintain different things.


hate apache as you like, anyone can configure it very quickly, since 
its all there.

and GUIs are a hell of a lot more complicated to maintain than text. 
 change one little thing in the file format and you've got to redesign 
alot of code... this doesn't happen with a file, since its being 
used directly.

Anyway, I'll try to list and add one or two comment lines for each 
available option in the next release, but I won't spend days writing 
docs for the config file.

if the config GUI is web based... then it relies on the server actually 
working...  but I'm not trying to argue with you, just pointing out 
the fact that server configuration is usually much better handled 
in text and I think many admins prefer it.  the fact that everything 
in windows is GUI based is the most annoying aspect about it.

I have a generic configuration managing library... the documentation 
is directly embeded in the configuration engine... if you save out 
the config or print it on screen, you have all the docs right there 
with it.  if you build a gui which uses the configuration data, you 
can also pull out the text from it. 


maybe that is what should be done.... allow string types within the 
config dialect (and store it appropriately).  then you/we could easily 
build tools using that info directly.

True, but the goal is to make it simple and easy to use for the average 
user, without requiring to read lengthly docs.

as long as the config GUI is only a tool over the files, and it doesn't 
overwrite the files automatically, I won't complain  :-)