AltME groups: search

I don't have time now to investiguate much further this issue, I'll 
have a look tonight.

Because I didn't had time to add code to support that feature which 
is probably very low priority currently. I'll probably add a config 
option to better control log storage for the v1.

Comet requires a connection to stay open, so it looks like this will 
take a Uniserve process for each open connection.


A Uniserve process (worker process) is not bound to a given client 
connection. Each worker accepts requests from ANY client connection, 
so it can handle hundreds of Comet-like connections with a few worker 
processes as long as each request doesn't take much time. For example, 
if each request takes 50ms, with 10 worker processes, you can handle 
200 req/sec.

Hmm, i think i did this some time ago.. pre Cheyenne days.

I remember googling a coding problem one time, and found a solution.. 
thought to myself.. "this guy knows the situatioh".. only to realize 
i was my own code i had posted some years earlier. ;()

This PHP code works fine..

<?php
 
 $password = "ClueCon";
 $port = "8021";
 $host = "127.0.0.1";
 
 function event_socket_create($host, $port, $password) {
     $fp = fsockopen($host, $port, $errno, $errdesc) 
       or die("Connection to $host failed");
     socket_set_blocking($fp,false);
     
     if ($fp) {
         while (!feof($fp)) {
            $buffer = fgets($fp, 1024);
            usleep(100); //allow time for reponse
            if (trim($buffer) == "Content-Type: auth/request") {
               fputs($fp, "auth $password\n\n");
               break;
            }
         }
         return $fp;
     }
     else {
         return false;
     }           
 }
 
 
 function event_socket_request($fp, $cmd) {
    
     if ($fp) {    
         fputs($fp, $cmd."\n\n");    
         usleep(100); //allow time for reponse
         
         $response = "";
         $i = 0;
         $contentlength = 0;
         while (!feof($fp)) {
            $buffer = fgets($fp, 4096);
            if ($contentlength > 0) {
               $response .= $buffer."<br>";
            }
            

            if ($contentlength == 0) { //if contentlenght is already don't process 
            again

                if (strlen(trim($buffer)) > 0) { //run only if buffer has content
                    $temparray = split(":", trim($buffer));
                    if ($temparray[0] == "Content-Length") {
                       $contentlength = trim($temparray[1]);
                    }
                }
            }
            
            usleep(100); //allow time for reponse
            

            //optional because of script timeout //don't let while loop become 
            endless
            if ($i > 10000) { break; } 
            
            if ($contentlength > 0) { //is contentlength set
                //stop reading if all content has been read.
                if (strlen($response) >= $contentlength) {  
                   break;
                }
            }
            $i++;
         }
         
         return $response;
     }
     else {
       echo "no handle";
     }
 }
 
 $fp = event_socket_create($host, $port, $password);
 $cmd = "api show dialplan";
 $response = event_socket_request($fp, $cmd);
 echo $response; 
 fclose($fp);

I'm coming to the conclusion that any communication other than via 
HTTP is a waste of time. Sockets suck.

I want to run another webapp on vps where I have site-assistant on. 
but site assistant is written in older version of cheyenne and I 
don't have time to port it to latest right now. that's why I am doing 
this

(and this new app is written in latest) .. maybe I should try to 
port it.. can't take taht much time

I don't think I managed to get two Cheyennes working at the same 
time properly.

I used to run several Cheyenne instance on linux server but it was 
a long time ago (v0.9.15 or 16). I didn't do much testing on that 
use case since then.

I'll have a look this weekend on this issue to see if I can get a 
quick fix (it wasn't simple last time I've checked, but the cold 
freezing wind blowing out here these days might help my brain be 
better irrigated ;-)).

It needs to be fixed anyway. I, sometimes, also may have needs to 
run different Cheyenne versions on the same production server at 
the same time. Graham also has needs for this, so I think it's time 
to settle this issue once and for all.

I'll have some free time from now 'til the beggining of next year 
that I intend to devote mainly to Cheyenne and CureCode, and if time 
permits, on some new frameworks.

Kaj: the bug you're referring to has been fixed in v0.9.19 released 
on 1st march 2009. Time to upgrade. ;-)

About multiple Cheyenne instances running at the same time, I'm finishing 
a new version supporting that feature. It took me more time that 
I thought first due to additional OS specific code required (wasted 
a lot of time on OS X). Still a few minor things to fix, tweak and 
test. It should be available by tomorrow.

@Doc: 

- I am not totally sure I correctly understand what you mean about 
webapp. I will need to process and try it some more.

- About code loading on on-page-start : yes I know of that, I am 
using it as dev setup so I know latest code is reloaded each pageload. 
I intend to set it on     on-application start when fully in production 
(But I have to admit the server is bgehaving really fast even with 
this setup all along)
- good catch, I will remove one simpauth.r

- I am aware of hosts file, but so far decided to edit the cfg file 
each time I switch ... which is in retrospect getting a little tiresome 
with more and more apps so I should think of  some naming convention 
for getting to all domains local and do it that way yes

- great for multi instance support . I also already ported site-assistant 
to 0919. it was much less work than I thought.. mostly just few things 
like do -> do/global and some 3 other tricks (no really aplication 
level changes were needed)

HAHA, looks like osx is a time waster, for Dock but for Carl as well 
8-)

Though Comet and Ajax can both deliver end-user experiences that 
provide desktop-like functionality and low user-perceived latency, 
only Web Sockets lives up to the promise of providing a native means 
to accurately and efficiently stream events to and from the browser 
with negligible latency. It is by far the most comprehensive solution 
for delivering real-time information over the Web. Not only does 
it provide full asynchronous duplex streaming communication with 
a single TCP/IP connection, but also benefits from few HTTP headers 
and more importantly allows the same message format to be used by 
both the browser and the origin service.

Still, I'm now considering the same as you. If I bundle Chrome with 
Syllable, by the time I am progressing with a few web apps, web sockets 
may have penetrated far enough to not bother with Comet

Terry: after taking the time to think about your comments regarding 
how web sockets should be integrated in UniServe & task-master, it 
starts to make sense and it inspires me on how it could be done and 
how the framework could look like. I'm writing a few specs on paper 
and will maybe implement a prototype tomorrow. This web sockets stuff 
is really going to be exiting after all. ;-)

well, is that not the time to carry on R3, Doc ?
You can use tasks advantageously for such job.

Stability (still alpha), feature-completeness (at least R2 level, 
including /library) and last but not least, time and money.

AFAIU but I might not have the latest news, Carl postponed full tasking 
support to R3.1, so there's still time.

So, if no devices have "work to do" then we sleep for a short period 
of time (as long as that period is greater than the minimum dispatch 
time for the host CPU.)

Henrik - yes, just read the discussion. Carl claimed, that RT has 
long time the code to IPC between processes. I objected, that no 
such code was standardised over-time, so that is maybe a reason, 
why ppl do wait for tasks. We await stuff like ipc:// scheme, etc.

Terry, wait to try the new web socket framework I'm working on. It 
should be ready tonight (if I find time to finish testing and debugging).

I have to make some time to play. Very cool Doc. Thanks for posting 
early and often.

Database accesses should be done in RSP scripts (because they are 
usually too time-consuming).

I think I'm lost (happens all the time).  Let's say I want to poll 
a remote web page for screen scraping (or remote php processing) 
every 2 seconds, and pass that to a specific port.. I would do this 
from the 'socket-app, no?

So, during the connection time, the target RSP script is not executed 
(unless you pass it a 'do-task in the 'on-connect handler). This 
target RSP gets associates with the opened socket until it's closed.

My demo is ready, but I found an issue to fix first with HTTP keepalive 
connections timeout colliding with web sockets timeout. The demo 
should be ready to try in a couple of hours (lunch time here).

When R3 features list will stabilize (beta stage), I'll make a more 
complete evaluation of the time required to port Cheyenne (lack of 
/Library can be workaround but will cost a significant amount of 
time).

I had hoped it would be in time for OSNews, but I thought I had better 
make use of the current slow news period. I'm not sure they would 
report on just some web server

That may be possible, but would be more complicated to support than 
web sockets as the server can't send data without getting a request. 
It will be hard to extend the web socket application framework without 
bloating it. Maybe a separated mod-comet would be a cleaner approach 
(but might duplicate a Iot of code in mod-socket). I will give it 
a look anyway, at least to estimate the time required to support 
it.

Ruby on Rails looks interesting, but not interesting enough for me 
to spend more than an hour 'playing' with it. BUT.. if RoR built 
some killer app that let me config it, I would take the time.

As I wrap up WIP3 for the current website redo, it seems a bit odd 
that each time, the WIP system gets smaller and better.

and you can cancel at any time, if you don't like it.

1) I tried -w before (I remembered you hinted me the last time to 
use it to always reload) but it didn't make the difference in this 
case (as far as I can tell).. does -w only work with source discributtions?

maybe I should better explain what I am doing so I won't steal your 
time here (and it's no problem basically as I will turn all global 
words that are only locally needed to something else) .. it's bad 
coding practica anyway

Funny thing is, if your "ATOMDB Query time:" changes randomly? That 
means someone else is running the test, and their results are being 
broadcast to your browser.
HOW COOL IS THAT!

Janko: I have in my todo list a full virtual system to add to Cheyenne 
allowing embedding webapps in a encapped Cheyenne. It can be done 
by replacing every filesystem accessing functions (DO, LOAD, READ, 
WRITE,...) by custom ones getting files from memory. The hard part 
is to integrate such approach within Cheyenne preserving perfomances 
for normal filesystem accesses while avoiding redundant code, this 
needs time for designing and prototyping.


I can't see an easy way to protect you webapps right now, but maybe 
other might have found a way to do that?

SVN r59 : experimental mod-upload released.  This new mod adds server-side 
API for querying realtime progress info on uploaded files. See the 
demo : http://localhost/upload.html(not commented yet).

Current restrictions:
- works only when posting one file at a time in a given <form>.

- can't make it work for IE (IE seems to have an issue with setTimeout( 
) method).

dinner time

Donation buttons added to Cheyenne web site at bottom if some of 
you feel inclined to help me spend more time on this project. ;-)

hmm.. your chat an mine came through at the same time.. must be the 
queueing

Yep, got both at the same time too. Could the lag between the button 
click and the response be reduced? I guess that having all the voice 
processing done locally (using a REBOL app) could make it realtime?

The first message takes some time.. every repeat is quicker.

after quite a long time

I tried to do the same text again, but no reponse this time

I thought this scenario could be recreated using WAIT or a loop that 
takes some time. 


Here'a another scenario.. a very typical websocket use..  A request 
comes in from the client "IBM".. the ws-script then checks a remote 
server continuously pushing a new IBM stock quote update, second 
by second forever?

Max: " the values of those words would be substituted when they are 
encountered."


I don't get where and how precisely this is supposed to happen (at 
Cheyenne boot time in %cheyenne.r like a preprocessor or at services 
install time in %misc/conf-parser.r as a dialect feature, and what 
could be substituted and what not, etc...). This does raise a lot 
of design questions and roughly looking, doesn't seem very different 
from what I was proposing. It seems that you'ld like the preprocessing 
occurs inside Cheyenne while I was more in favour of an external 
preprocessor script as I'm not sure how often users will need such 
feature.

I was thinking that it could be added within the dialect... I might 
look how to add it myself and I could give you an example... maybe 
not now cause I have a lot of things on my plate, but the next time 
I go deep into cheyenne's guts (next time I work on mod-remark).

do any of the rebol loops have 30 second time-outs?? very strange

yeah, I need an async https port.. still getting overlapping tts 
broadcasts when two or more people submit at thesame time.

huh, I have to comment something.. I went looking what curecode looks 
like .. and founc this one http://curecode.org/rebol3/view-tickets.rsp
.. when I am clicking links the page reloaded so quickly that I couldn't 
determine if you are doing an ayax call or the whole page is reloading 
and I can't see it.. I also noticed this on my cheyenne projects. 


I don't get how you managed to do this. There are numerous servers 
outthere and rebol is not the fastest language and I haven't seen 
something like that anywhere?!?!? I mean I think also the static 
pages that you load from some webserver take some more time.. ?

I would answer, that spending my time reimplementing things like 
/Library, CALL or hash! in R3 using host kit (and giving my rights 
on it away), while code for such features already exists in R2, would 
be extremely stupid from me.

Btw, to clarify also other things, "switching to R3" by porting Cheyenne 
(which by itself would took weeks), would put me in difficult position 
where I would have to maintain 2 versions of Cheyenne (R3 version 
and R2 for all my installed personal and business webapps). Making 
a full switch (porting all my web and native apps to R3), would require 
*months* of work. I currently have around 12h a week (7 days) of 
free time for non-business projects. So, "switching" to R3, would 
mean stopping all evolutions on all my products during months, maybe 
a year until I can port everything to R3. This is not an option for 
me.


So, only a "R2 feature-complete" R3 version could make that doable 
in an acceptable time frame (and with much less risks of regressions). 


And all this huge work for what gain? Well, without threading, almost 
none (to be fair, maybe a little bit lighter source code base).

I've never need to wrap a C library, but I have to wrap libraries 
written in other languages all the time. Languages without C interfaces.

I was looking at using some of the windows speech recognition stuff 
via /library  earlier today. Accept voice prompts, and push to a 
web page via sockets.

Playing with PowerShell for the first time as well.  Has some promise.
The speech recognition in Windows 7 is world class.

time for some debugging

It seems to me that there's a bit of misunderstanding here on how 
OSS works. My understanding is that peoples are willing to contribute, 
because they *need* something and they *can* obtain it by modifying 
the source code. Expecting that your product development forces will 
magically increase because you're going open source is, in the general 
case, an illusion. People will contribute only if they're interested 
in your product, have a need to fullfill, and time/skills to make 
it. So, a critical mass of users is required to get enough contributions. 
If you don't get enough contributions, don't blame it on the open 
source approach, blame it on your product (or on your communication), 
because it doesn't attrack enough people to reach the required critical 
mass.

Just an observation that the developer base shrinks as time progresses 
...and I doubt anything apart from open sourcing r3 will help

And move-file won't be fast all the time. e.g. if you're moving the 
file to another drive.

Ok, no problem... will add content-length field. Shouldn't take much 
time.

Some texts (that are part of application) are a little longer. I 
was thinking of replacing all of them with "tags", like "add_now" 
instead of "Add now" and "warning_text" instead of "bla bla ........................" 
and having the default language as this tag language from which it 
is then translated to english, slovene, etc.. anyone has any thoughts 
about this?


I could also make a tool that would statically replace all the values 
in say "..." and #[...] if I decide to do so at one time. That's 
why I would preferr the same system for most of things.


I will probably have a separate files where the whole page is text 
, like the instructions.

(back from core :)

some nice results with map!


>> length? n  ; n is a map containing random keys, and a 25 byte 
string value
== 2000000


loadit 1000000  ; loads in an additional 1m key value pairs (same 
as others, and returns execution time
== 0:00:9.27  


test3 1000000   ; test3 is a function that GETS the value of a particular 
key 1M times, returns the last iteration's value (!string),  returns 
execution time
this is a bit of a string
0:00:01.616

That is blazing, nearly 10 times faster than Redis

Kaj: "From earlier messages here, it was my understanding that Cheyenne 
on Linux can run its UniServe taskmaster processes under a non-root 
user account by enabling 'userdir in the modules and 'user and 'group 
in the globals. However, when I do that, all of Cheyenne is still 
running as root. The designated user and group exist. What else do 
I need to do?"


It was working last time I've checked. What Cheyenne version are 
you using? Have you remove all the log files generated by Cheyenne 
that are own by root before testing? Did you tried with other combinations 
of user and group?

Yeah, it was worth spending some time on it

What do you mean by that? Cheyenne's UniServe task masters run in 
separate processes, so when one request takes time other task masters 
handle other requests, but within one request your code is executed 
serially, the way you program it. REBOL is single-tasking, after 
all

I start it up with sudo under linux and the very first time it works.
I reboot, and then try again, and it just quits ... everytime.

I then use rebol/core and when I do %cheyenne.r it complains it can't 
write the cache efs file.  

I allow it, and then restart the cheyenne binary and then it runs 
okay.

So my question is, does the cheyenne binary need to write the cache 
efs file each time?

What do I need to do to allow Cheyenne to start reliably each time 
...?

going to copy the .cache.efs file to cache.efs and then copy it back 
each time with a shell script to see if that fixes it

Well, I don't know what is going on but hopefully my saving a copy 
of the .cache.efs file and copying it anew each time the user restarts 
cheyenne will bypass this issue

I use isolated approach too, but I use sqlite databases which are 
not that good for webapps but ok for isolated since 1 user wont make 
hundrets of connections to the same database at the same time

the time gained is in writing .. not execution :)

Sure .. I understand that for interpreted languages, there are also 
performance downsides .. but the gain is in the time to deliver product

That's why people use interpreted languages/scripting languages .. 
 to speed up the development cycle not for writing time critical 
apps

long time I haven't looked at Da Vinci...

Websocket server code can be run from two places: either in Cheyenne 
main process (allows accessing all clients ports and detecting ports 
open/close events) or in RSP scripts (using 'do-task function from 
a websocket app) when the job takes too much time (like accessing 
a database).

scratch the word "using" in that last post.
Post posting.. a great time to proofread ;)

exactly. I tried that tonight as I look at cheyenne for the first 
time. However, the on-page-start/end do not seem to work (in debug 
mode). The instructions are so simple I wonder why it would not work 
though.

I mean this example code in the app-init.r: 

 on-page-start: does [
        set 'start-time now/time/precise
    ]

    on-page-end: has [pos elapsed][
        if pos: find response/buffer "</body>" [
            elapsed: now/time/precise - start-time
            insert pos reform ["<br>Processed in :" elapsed]
    ]

45 minutes? I outta go. I messed up your time.

:) We need to write some code then. See ya. And thanks for the time.

I don't find much use of rebol view. I'm interested in browser based 
stuff, and that's the only way I'd spend time with rebol. I'd love 
to take a look at Temple.

Dockimbel, will you provide an encapped version again? Last time 
I had quite a mess figuring out what Cheyenne version worked with 
what REBOL version, so I just avoid upgrades unless there is an encapped 
version.

damn...shouldn't chat on more than 3 windows at the same time...:-)

Anton, for me right-click copy only works the first time in AltME. 
Then it stops working.

I'd like to make that side proc an option.  Holger added it for async 
DNS lookup, but a great deal of the time no lookup is being done, 
or sync DNS lookup is just fine.

The 2.7.8 planning seems to include some goals in that direction, 
making View not bomb out when X11 is not running, but the time plan 
was abandoned

I am not, but last time I "felt" an Include conflict, it was very 
easy to resolve, in fact

I'll have to do some more digging Doc, and maybe see what do* does 
since I haven't tried that. I'll actually read some source if need 
be. ;-) 


What I saw with your new change is something I also saw once with 
the previous approach. Plain CGI worked fine until an RSP was run, 
then the CGI failed, *then* both the test RSP and test RSP web app 
started returning with garbage at the top of the page. More garbage 
each time, like a buffer was being added to. Then the plain CGI worked 
again. Doing a reset on the workers solves the garbage problem.


My Cheyenne EXE is from 2009 so I'll check for a new one before doing 
anything else.

From 2009 -> more than 90 revisions since then, these issues have 
been solved a long time ago :-)

Are you using the official binary v0.9.19?

So, finally... it's time that I learn how to use the RSP API.

Long ago, there was a RESET function in REBOL.  It would restore 
the runtime environment without reloading the exe.


This allowed REBOL to run as an apache mod, because the process would 
persist and could start "clean" each time.

Such a thing could be done in R3.

Hi Doc, nice list. But would like to ask you, what about a way how 
to terminate long time unused worker processes? They are started 
automaticaly when needed, but there is no way how to stop them when 
the server has no activity.

I've been releasing cmd encapped binaries for cheyenne for a long 
time now

Peter: the error you get is caused by trying to run multiple sessions 
of Cheyenne at the same time, see my post at Thu 19:04 about that.

This is a debug print of a correct response:
req/result [

    done [reply seq 1 service "Generic REBOL Service" commands 1 time 
    0:00] 
    ok [date 9-Nov-2010/2:16:33-7:00]
]