AltME groups: search

'do-task sends the argument string to a background worker process, 
so you can do whatever you want with it without blocking the main 
server. The target used is the initial RSP script (ws.rsp in the 
example).

do-task allows you to have multitasking in you web socket application. 
Internally, it  re-uses the task-master service and the RSP IPC system. 
That way, you can easily integrate one or several web socket service 
or applications in existing RSP scripts or webapps.

so do-task in your example would simply echo back to the browser?

do-task in this example, sends the raw incoming data from the browser 
to a worker process using ws.rsp script to generate a response that 
is sent back to the web socket app (if a /on-done callback has been 
defined) or directly to the client browser.

The /on-done callback is for post-processing the response and decide 
what to do with it. It will be available in the new revision I'm 
commiting right now.

SVN r50

FEAT: /on-done callback support added to 'do-task function in socket 
application.

FIX: socket application API now has more homogeneous argument passing 
(client port always first now).

Thanks, it's fun to do and will probably be very useful for all Cheyenne's 
users.

Just fooling around.. there's some interesting behviour here... 

changed the on-timer function in  ws-test-app.r file to... 

	on-timer: does [								
		foreach port clients [						 
			out: do %test.r
			send/with out  port				 
		]										 
	]


.. and created a test.r file in Cheyenne's root thus.. 

rebol []

a: "42"
b: "43"

out1: rejoin ["OUT1: " b]
out2: rejoin ["out2: " a]
out1

Graham: I have a client application using a similar method to do 
updates.
Once the update exe is downloaded :

write/binary %application-maj.exe decompress res
wait 0.5						
call "application-maj.exe"						
quit

Then, the first thing the application does when started is :

if find system/options/boot "application-maj" [
	delete %application.exe
	call/wait "copy application-maj.exe application.exe"
	wait 0.5
	call "application.exe"
	quit
]

if exists? %application-maj.exe [
	attempt [delete %application-maj.exe]
]

The other way I used to do it was maintain binary diffs and xor them 
back to update the main binary .. but again that requires the user 
not to run two copies :(

PHP: quite stable as long as you don't do live %httpd.cfg file reload 
(mess up PHP processes).

do-task in RSP : no, UniServe's model is : one unique dispatcher 
process, one or many worker process.

Timers: no, they require an event loop, worker processes can't have 
one because they are supposed to work in a request/response model.

Btw, in the current SVN revision, it's not possible to call 'do-task 
from 'on-timer event.

The current framework gives you access to client ports value, RSP 
session object associated with a client, RSP script execution (with 
'do-task).

I think I'm lost (happens all the time).  Let's say I want to poll 
a remote web page for screen scraping (or remote php processing) 
every 2 seconds, and pass that to a specific port.. I would do this 
from the 'socket-app, no?

You would generate the timer events in the socket-app, pass the job 
to do to an RSP script, and use the /on-done handler to send the 
response to the client you want once the RSP script finished its 
work.


(this will work once I fix the issue with 'do-task calls from 'on-timer 
events).

SVN r51
FEAT: /ws-apps folders are now protected from direct access.

FEAT: modified socket apps source files are reloaded and used by 
new clients.

FIX:  web socket application's 'session object renamed to the more 
accurate 'rsp-session.
FIX:  'do-task calls now supported in 'on-timer handler.
FIX:  scheduler issue with deleted jobs.

/on-done installs a callback handler (block! or function!) that gets 
invoked when the job sent by 'do-task to the RSP script is finished.

How do you pass a do-task to an rsp script? (what variables are passed?)

I've fixed several issues with timers, so yes probably. I still don't 
get why you want to "do" a script on each event...

Here's a modified on-message from ws-test-app.r 

	on-message: func [client data][						
		;send data
					
		do-task/on-done data func [client data][	 
		

   ;data: do/args %/c/cheyenne/cheyenne/test.r {one "two three"}		  
       ;doing a script passing /args

   data: read http://www.php.net/usage.php				      ;as 
   a proxy to PHP server
			
			send/with data client
		]
	]

what is missing in R3 to have Cheyenne working on it? we should make 
sure Carl doesn't forget any feature needed, I sure wouldn't like 
being stuck with R2 because Cheyenne wont run on R3, do you have 
a list that we should relay to BrianH who will relay it to Carl ? 
I know it's still early, but having a plan should help

For web socket apps, the advantage would be to not have to require 
'do-task to use the sync version of these protocols from a worker 
process. OTOH, that would put more burden on the main process lowering 
its scalability. So I've not decided yet, this would probably require 
good benchmarking first.

aha, I see it now..


btw: I noticed one thing.. when I uploaded file request/conetnt keys 
returned to strings from words .. so I had to do req: request/content 
forskip req 2 [ change req to-word first req ]  to fix them back

Yup, I agree, at least the parts that need the web to do (which isn't 
much on that list).

Now all you need to do is come up with the killer app :)

So, to copy this method into Cheyenne websockets, i need to 'DO' 
.r scripts.. not rsp.

I use Amazon EC2 and it's reliable to a point .. if it goes down 
I just start another instance ... and in 6 months, I've only had 
to do that once.

Do they allow you to custom config?

I just found out about a bug I was making and could only be discovered 
when cheyenne was first started, because some word was undefined 
then so it triggered an error, if I did some other things I couldn't 
see it but it would return wrong information in reality .. 


1. is there any way maybe that would make cheyenne "reset" each request 
(for debugging & testing purposes, not production) .. the -w option 
seems not to do this. 


this could probebly be solved by wrapping also those parts in functions 
... another question..


2. Is there any way I could see all globals I created so I could 
find wtich ones I "leaked" unintentionally .. 


it would be very helpfull if rebol could let you define functs that 
would warn you if you used or defined any global words

What woudl be the best way to do some casual source code protection 
of cheyenne webapp if you want to distributte it as a "download and 
install" option. I guess I could make some sort of code obfuscator 
that would change the names to something without meaning, is there 
anything better one could do?

Janko: I have in my todo list a full virtual system to add to Cheyenne 
allowing embedding webapps in a encapped Cheyenne. It can be done 
by replacing every filesystem accessing functions (DO, LOAD, READ, 
WRITE,...) by custom ones getting files from memory. The hard part 
is to integrate such approach within Cheyenne preserving perfomances 
for normal filesystem accesses while avoiding redundant code, this 
needs time for designing and prototyping.


I can't see an easy way to protect you webapps right now, but maybe 
other might have found a way to do that?

The only way I've heard of to do WebSocket in older browsers (almost 
all of them right now) is the Flash implementation

I think not, older brosers only have xmlhttp (ajax) option, iframes 
and other comet like tricks but they can't do a socket .. I haven't 
seen what js.io does.. if you wanted to make a "pure" html chat without 
comet or polling you made JS that talked to invisible 1px flash and 
flash has XMLSocket or something like that. But I heard recently 
that it can be used for other things than xml ..

I think it's official.. Flash is dead.


I was modifying some flash xmlsocket based apps to work with Cheyenne 
websockets,  and then realized, why bother? I can do it all with 
HTML 5

HTML 5 has.. 
 
CANVAS AND VIDEO

http://www.youtube.com/watch?v=Kdf0x8am1cg&feature=player_embedded#
chroma key effect. 

http://www.youtube.com/watch?v=ZN_r9jxL0-A&feature=player_embedded

LOCAL STORAGE
Chrome has SQLite built in, accessible via JS

Flash might move to RIA (but who's interested to do real apps in 
Flash?) or - many mp3 etc vendors are doing their UI using flash. 
As for html 5, we are still talking vapor, unless it is supported 
by most browsers, which are used by most ppl IE still has 62% of 
share, and IIRC CSS3 and HTML5 are going to be supported in IE9. 
How long do you think will it take to replace those 60% of IEs out 
there by IE9? Well ... I know what you are trying to say ... it is 
inevitable ... but ... not yet, not yet :-) .... this probably belongs 
to advocacy though ....

who's interested to do real apps in Flash?
 http://www.adobe.com/products/flex/buzz/customers/list.html

There's a blur with websockets.. traditional usage takes a form, 
serialize and send to the server as a associative array in the url.. 
With sockets, that's not necessary. 

Forms are becoming less relevant.. As individual fields can do their 
own thing.. 

<input type="text" onblur="ws.send(this.val());">

It's great if the client won't primarily be a browser, as long as 
you stick to full dialects and never DO the code.

Fortunately you can't create reference loops without DO, so if you 
stick to LOAD/all you should be fine.

Hey Doc, refreshing the page after every change to a ws-app is painful. 
 Can't we just DO a script using do-task/on-done, put some error 
checking on the front end? So that the ws-app is just a handler of 
sorts as well.

could then set up a timer/schduler.. and makes changes live.. that 
would  make script development 10x faster. 
Refreshing to reload ws app is like a compiler.

This works in answer to my last questions.. 

	on-message: func [client data][
		;-- escape all html tags for security concerns
		data: copy data
		replace/all data "<" "&lt;"
		replace/all data ">" "&gt;"

		params: load copy data  
			
		do-task/on-done data func [client data][
		 n: do load %test2.r
		 broadcast out
		]
	]

If i put a "wait random 10" line within the do-task/on-done function. 
that returns an alert back to the page, then open a couple of browsers.. 
and fire the function in both, I get a lot of strange behaviour.. 
and I'm not convinced they don't block each other?

Doc: if there's any small thing we can do on our side in 2.7.8 to 
help with SU situation, let me know.  As you know, running the server 
as SU will be a problem for many folks.

I tried to do the same text again, but no reponse this time

Hmm ... what do you mean? You normally run your CGI scripts, which 
can connect to SQL database, no? Dunno, how it is done in case of 
persisent connections ...

if you create an application (this requires some knowledge on how 
to do that) you shouldn't need to do that other than during the startup 
sequence, and then close the connection again during close of the 
application.

you can do this without apps, if you want, but then you have to manage 
startup yourself. you can see variable persistence in a page using 
a page like this:

<%
if not value? 'x [x: 0]
print x: x + 1
%>

In addition to Henrik's answers, database access can be used in two 
ways, either "manually", opening and closing connection yourself 
from a CGI or RSP script (READ mysql:// prefered in that case), or 
by relying on the RSP webapp API to manage persistent connections 
(see DO-SQL function in RSP API doc).


In both cases, you need to load mysql-protocol.r. The best place 
for that is in the config file in GLOBALS section, just add :

worker-libs [
    %<path-to-mysql-file>/mysql-protocol.r
]


Usually, I define a %libs/ sub-folder in Cheyenne's folder and store 
there all the libraries used by all my RSP or CGI scripts.

Hey doc, how can we handle loops in within do-task/on-done ?
Anything i try seems to block and cause port issues.

How about this scenario.. an on-message comes in, the code then does 
a 'read on an external server.. but the server takes 30 secs to respond.. 
does that blocK?


READ is a blocking operation, so it should not be used in Cheyenne 
main process (where ws apps run). Two possible solutions :


1) Use async HTTP with callbacks (I've proposed to include in Cheyenne 
the one I'm using currently for other products).

2) Send the READ job to a worker process using DO-TASK (but it will 
block it for 30secs, so not a very scalable solution).

I would use 1).

Would you put a timer in the do-task/on-done func?


Not sure what you want to achieve precisely there. In your "READ 
quotes from remote server and send them every sec" scenario, I wouldn't 
use do-task, but async HTTP (much more efficient and scalable). I 
think I definitely need to add a few async schemes and make an example 
app.

2) Send the READ job to a worker process using DO-TASK (but it will 
block it for 30secs, so not a very scalable solution).

Tried this.. it's not very ws:// "broadcast" friendly

huh, I have to comment something.. I went looking what curecode looks 
like .. and founc this one http://curecode.org/rebol3/view-tickets.rsp
.. when I am clicking links the page reloaded so quickly that I couldn't 
determine if you are doing an ayax call or the whole page is reloading 
and I can't see it.. I also noticed this on my cheyenne projects. 


I don't get how you managed to do this. There are numerous servers 
outthere and rebol is not the fastest language and I haven't seen 
something like that anywhere?!?!? I mean I think also the static 
pages that you load from some webserver take some more time.. ?

Terry: fully open sourcing R3 and using a dual licensing (GPL + commercial), 
as a lot of other small companies do, seems to me like the only viable 
model when you don't have money to invest.

Maybe we should do REBOL community reunion, even without RT, who 
knows when next devcon is going to happen. I would like to see you 
guys, before we close the shop here :-)

ok ... I guess I need to do another encap

I was reluctant to do that before, CureCode wasn't able to run multiple 
instances of itself on the same Cheyenne. Since 0.9.10, it's fixed, 
so I'm in the process of collecting all my current CC instances scattered 
around on multiple servers on the same Cheyenne server.

Why? Do you intend to read the source code and be happy with that? 
What good does it do, if it doesn't reduce the number of bugs, vastly 
improve the design or in other ways lift R3 beyond where it is now?

Do you think the curecode bugs would then magically be fixed?

 Do you really think that this is my intent by having it open sourced? 
 It's open source because its the logical way to distribute such applications 
 nowadays, offering to any user full insight and control over the 
 product.

Usually do save it to disc ...

Thanks Doc, I did get it running with some simple selects and updates 
now I just need to learn how to format the returned data. This is 
good stuff. By the way I am using the do-sql method and works great.

James: it's not mandatory, you can load mysql driver from 'on-application-start 
event for example (but you'll need to do that for all your webapps). 
It's just simplier to declare it in config file once for all.

I can write a CGI to do that, but it will be slower because it requires 
an extra CALL from cheyenne.

what to do to be saved in logs/default-access.txt user-agent and 
referer ?

I agree that #[...] really makes things easier and cleaner. Performance 
impact is not significant, the overhead is just a string! PICK in 
a block with a static index as argument (that's one of the nice advantage 
of compiling RSP scripts). OTOH, the SAY function has to do a hash 
table lookup to get the right index (so a tiny little bit slower 
than the literal form).

Some texts (that are part of application) are a little longer. I 
was thinking of replacing all of them with "tags", like "add_now" 
instead of "Add now" and "warning_text" instead of "bla bla ........................" 
and having the default language as this tag language from which it 
is then translated to english, slovene, etc.. anyone has any thoughts 
about this?


I could also make a tool that would statically replace all the values 
in say "..." and #[...] if I decide to do so at one time. That's 
why I would preferr the same system for most of things.


I will probably have a separate files where the whole page is text 
, like the instructions.

Do I need to configure any paths to get it up & running without encapping?

is there any other configuration or something to use databases with 
cheyenne? I configured as it shown in web site but I always get the 
error:

##RSP Script Error: 
	URL  = /
	File = www/index.rsp
	** Access Error : Invalid port spec: odbc://test2
	** Where: do-sql 
	** Near:  [do-sql 'test2 "select * from table" 
__txt 62
] 

my rsp site is:
<%
	print do-sql 'test1 "select top 10 * from sec_log1"
%>

and the httpd.conf file:

databases [
	test1	mysql://[root-:-localhost]/test
	test2	odbc://test
]

Add %path/to/mysql-protocol.r to your WORKER-LIBS block in the %httpd.cfg, 
that should do the trick.

I see. I don't have SDK. Is there a license issue about mysql/odbc 
drivers or do you have a plan to put in next releases?

Doc, any explanation as to why it can be so hard to start Cheyenne? 
 I think it is to do with the .cache.efs file

The designated user and group exist. What else do I need to do?

<rant>One thing Cheyenne (and Rebol) needs to do is grow up, and 
lose Altme as the primary source of communication</rant>

I'm thinking about porting  Redis to Rebol as a datastore for Cheyenne, 
but I'm not sure of performance.. 

Redis is a key/value db like CouchDB, with a number of nice features 
(persistence etc.)


Now in Redis, I can create a db with 120,000 key / value pairs.. 
keys are integers from 1 - 120,000 (in this example) with a 10 byte 
string as the value

In redis, i can do 10,000 GETS (return the value of the key) in 1.39 
seconds or 7200 / second.


Now trying to model this in Rebol, i used R3 on a newer machine (i7 
/ 6gb ram) with a block of integers from 1 - 120,000

PICK is blazing:  10, 000 picks in .011 seconds
but with SELECT it's 10,000 GETS in 2.23 seconds


My question is..  is there a faster way in Rebol to select a value 
from a key?

Do you have an map-each example to try the problem above Henrik?

<rant>One thing Cheyenne (and Rebol) needs to do is grow up, and 
lose Altme as the primary source of communication</rant>


AltMe is convenient because almost all Cheyenne users come here for 
realtime chatting, but I agree on your comment. Adding a web forum 
to the Cheyenne web site would be a good thing.

Kaj: "From earlier messages here, it was my understanding that Cheyenne 
on Linux can run its UniServe taskmaster processes under a non-root 
user account by enabling 'userdir in the modules and 'user and 'group 
in the globals. However, when I do that, all of Cheyenne is still 
running as root. The designated user and group exist. What else do 
I need to do?"


It was working last time I've checked. What Cheyenne version are 
you using? Have you remove all the log files generated by Cheyenne 
that are own by root before testing? Did you tried with other combinations 
of user and group?

I get an "confirm" error from a RSP page that tries to delete a local 
file. How can I do this?

What do you get when trying to delete this file from console launched 
with the same uid/gid than Cheyenne?

The to do list for 2.7.8 mentions fixing the related error message, 
so I suspect Carl has found that this doesn't work when he briefly 
tried to run Cheyenne on his new server

--- mod-userdir.r.original	2010-05-09 19:28:10.000000000 +0200
+++ mod-userdir.r	2010-05-11 00:45:24.000000000 +0200
@@ -12,40 +12,81 @@
 	on-started: does [do boot-code]
 	on-reload:  does [clear boot-code]
 	
-	get-ugid: func [name [string!] /local file uid gid][
-		if none? attempt [file: read %/etc/passwd][
+	get-ugid: func [name [string!] /local file line uid gid][
+		unless attempt [file: read/lines %/etc/passwd][
 			log/error "accessing /etc/passwd failed"
 			return none
 		]
-		unless parse/all file [
-			thru name 2 [thru col]
-			copy uid to col skip
-			copy gid to col
-			to end
-		][
-			log/error "reading /etc/passwd failed"
+		foreach line file [
+			if all [line: find/case/match line name  col = first line][
+				return either parse/all next line [
+					thru col
+					copy uid to col skip
+					copy gid to col
+					to end
+				][
+					reduce [to-integer uid to-integer gid]
+				][
+					log/error "invalid format reading /etc/passwd !"
+					none
+				]
+			]
+		]
+		log/error "user not found in /etc/passwd"
+		none
+	]
+	
+	get-gid: func [name [string!] /local file line gid][
+		unless attempt [file: read/lines %/etc/group][
+			log/error "accessing /etc/group failed"
 			return none
 		]
-		reduce [to-integer uid to-integer gid]
+		foreach line file [
+			if all [line: find/case/match line name  col = first line][
+				return either parse/all next line [
+					thru col
+					copy gid to col
+					to end
+				][
+					to-integer gid
+				][
+					log/error "invalid format reading /etc/group !"
+					none
+				]
+			]
+		]
+		log/error "group not found in /etc/group"
+		none
 	]
 	
-	change-id: func [id [word! integer!] /user /group][
-		if word? id [
-			if none? id: get-ugid mold id [return none]
-			id: pick id to-logic user

+	change-id: func [id [string! integer!] /user /group /local gid][
+		either string? id [
+			unless id: get-ugid id [return none]
+			set [id gid] id
+		][
+			gid: id
 		]
-		either user [
+		if group [setgid gid]
+		if user [
 			;logger/file.log: join logger/file ["-" id %.log]
 			setuid id
-		][setgid id]
+		]
+	]
+	
+	change-gid: func [id [string! integer!]][
+		if string? id [
+			unless id: get-gid id [return none]
+		]
+		setgid id
 	]
 	
 	words: [
 		user: [word! | integer!] in globals do [
-			repend boot-code ['change-id/user to-lit-word args/1]

+			repend boot-code either word? args/1 [['change-id/user/group 
mold args/1]] [['change-id/user args/1]]
 		]
 		group: [word! | integer!] in globals do [
-			repend boot-code ['change-id/group to-lit-word args/1]
+			unless empty? boot-code [change boot-code [change-id/user]]

+			insert boot-code reduce ['change-gid either word? args/1 [mold 
args/1][args/1]]
 		]
 	]
 ]
\ No newline at end of file

I'm not sure how meaningful it is to also specify a separate group 
(different from the group of the user), but if you do it should be 
looked up in the group file instead of the users file, so now it 
does that

can I somehow automatically execute run one file or a pack of code 
before each page load If I am not in webapp? I added do %... to each 
html file now but it's not very elegant.

So my question is, does the cheyenne binary need to write the cache 
efs file each time?

What do I need to do to allow Cheyenne to start reliably each time 
...?

I then do a ps aux | grep cheyenne to confirm that it is no longer 
running

So, what advantages do you see being lost with a port to the JVM 
or .Net ?

i know, you're just against being the person to do it: :)

I could do some hotfix to CureCode too if required

Just to allow users to add examples of how to use functions etc .. 
since many people do not have access to the various "wiki"s

is there a ws:// protocol implementation for R2? How do I connect 
to a server and use websocket without a browser?

As this is running in main process, when

;-- any handler runs, it will block the server, so, you have to keep 
your code

;-- very efficient, it should run in between 1ms and 10ms if you 
want your Cheyenne

;-- server be able to scale to hundreds of concurrent clients. That's 
the cost to

;-- pay for not having multi-threading...Anyway, you can use the 
'do-task function
;-- to run longer code without blocking.

I use websockets for passing javascript and JSON to a static page. 
 The pages start off with a single javascript function, then i use 
websockets to lazy load.


Here's  pseudo for an email client using Cy (shorting Cheyenne to 
Cy from C.. keep confusing myself)


- Start with a blank page with the websocket function. No HTML, No 
css.. link to Jquery only  .. this page is under 1k

- Send an initial message to Cy with the page name, Cy responds by 
pushing the necessary HTML (simple email form), CSS and JS (a single 
function to send back the form results to Cy) to the DOM

- User fills out form, and submits.. pushing an array (JSON, string.. 
whatever) with the message details to Cy
- Cy sends the message via rebol mail

... but then 


.. Cy polls the users pop mail acount on a regular basis, and upon 
finding a new message, sends <i> sends a javascript function</i> 
to the page, that gets EVAL 'd . This could do some animation in 
a canvas tag, push some message to the browser's chrome.. whatever.
 

All using the same tiny webpage. When you look at the source, you 
see 20 lines of code.

Sure .. just wondering if it's easier to do thru web ports and if 
the server can notify me thru an open port when it succeeds or fails 
rather than me poll the server

well remark allows you to do this on ther server... and you could 
cache it so its just processed once.

Graham: why do you need to edit httpd.cfg for each user?

The Syllable Server manual has an example how to do that on Linux:

Graham: domain or maybe subdomain? if subdomain and maybe domain 
also you could probably put nginx in front of cheyenne and use rewrite 
rules to do this (depending on the concrete situation)

Who do you use to manage your domains?  Godaddy?

Cheyenne is supposed to be able to reload the configuration without 
having to restart so I just need to be able to tell it to do this

There's some info about it in the changelog. Maybe it says how to 
do it on Windows