AltME groups: search

I have been doing selects and have about 1% failure rate...

I was thinking there might be a timeout value which is set a little 
too short, but I am such a neophyte when it comes to tweaking ports 
and with mySql in general...

hum, I am calling  wait  0.01  pretty often within the script to 
allow the asynchronous rebol core some breathing time which REALLY 
rocks btw... scanning multiple dirs, while letting the UI and mySql 
to work at the same time , I'd not be able to get it running so easily 
in any other language.... :-)

I have Cheyenne and UniServe releases pending in the pipe, mysql:// 
update is next on my list.

But the Win server and the client are on the LAN 100Mbits

Right, the problem is to get connected to the server. Sometimes, 
the server reject the connection attempt. If you successfully connect 
and keep the connection opened, there should be no problem AFAIK.

Got a problem where the server app while in a wait state jumps to 
50% cpu, and memory jumps from 4Mb to 30Mb .. using rugby.

Gabriele, I do exactly the same kind of work arounds, in my apps, 
I just highlight the panes which have failed and include a user-driven 
"refresh button"  but this is not very elegent.. nor is requerying 
until no error is returned...  although I was thinking of adding 
a persistent requery in my  top-level query function.

The server I am accessing is very quick and in general, it accomplishes 
its query in micro seconds.  so its not really a question of server 
load...

Doc.   my very competent db admin, told me that it could simply be 
an issue that the client is expecting a reply from the sql server 
too quickly and that it is likely a simple timeout issue.  HTH.  
 if there is a way to let the tcp wait a bit more, it could be that 
the fraction of a second more needed might be all that is required.

Doc,   for my useage, the access denied error happens 100% randomly. 
   more connections only mean more chances of getting the error, 
but I've had it fail on the very first connection attempt and have 
had several thousand go without errors...   then I'll have 5 within 
a 100 ... there really is no pattern I can detect.

my current app is so threaded because of the async core that its 
impressed everyone at the office... I can be scrolling a list view, 
reading files doing sql queries and reading directory lists which 
buildup view menus in real time... everything is fluding and strangely, 
having sql and a loop browsing through several hundred directories 
scanning their content is almost unnoticeable !!!!!!    scrolling 
speed goes down by maybe 20%... but network access remains pretty 
steady... congrats to RT...

Looking at my mysql:// code ('do-handshake and 'read-packet functions), 
I notice that I didn't use any 'wait to synchronize with the server. 
Instead it loops on 'read-io until it gets all the data expected 
which may waste some cpu time. Looking today, I'll say that's not 
the best way to do that, and the code stability could benefit from 
a couple of 'wait calls. I should definitely rewrite the low level 
stuff.

Maxim: if you're guessed right on the access denied issue, you should 
be able to fix it by addind a few 'wait 0.01 to the 'do-handshake 
function. Put one 'wait just before each call to 'read-packet and 
if you have time do some tests.

I am having problems connecting to mysql on windows XP. The error 
message I get is "Client does not support authentication protocol". 
I'm using localhost as the host and have granted all priviliges for 
['user''-:-'localhost'']  identified by ''password' .... etc. I'm used 
to linux, and it''s been

hmm.. and you are using Mysql-prot.r ?

I've never changed it.. and it's worked fine..

There are some gotchas using 'localhost'. Now on my linux machine, 
when I changed my I.P. from 192.168.1.1 to 192.168.1.6, I then had 
to connect using my machine's hostname (which was "linus") . If I 
attempt to connect using this machine's hostname (lucy) I get : MYSQL 
ERROR 1045 : Access denied for user ['tim'-:-'lucy'] (using password: 
YES). And I *have* also granted privileges to [tim-:-lucy] indentified 
by 'password''

Tim: MySQL 4.1 uses a new encryption method for user passwords, try 
add a new user assigning the password with the function OLD_PASSWORD('pass') 
rather than PASSWORD('pass'), and attempt   open mysql:// ... etc. 
 with the new user. Hope this helps.

I googled it and added the user as per the OLD-Password syntax. Now 
I tried a: read join mysql://tim:[password-:-localhost]/hs4u "show tables"
and got the same error message

anyway, have a look at http://www.compkarori.com/vanilla/display/peek_and_poke.r

Apologies on Insert -- I'd assumed from the fact you'd asked the 
question that MySQL did not support the Values syntax.


Update is trickier as you are potentially changing a whole load of 
values.

Best way is to have the update data in a 2nd table, and then select 
as appropriate.

I have a table with some Products.

I need to copy each line from the Products table in a Command table 
(with a different structure, but same key) IF THEY ARE NOT ALREADY 
THERE, and add a quantity value.

I think I mean 
   where prod-code in
not NOT in
(the code is not tested, and may not be possible in mySQL)

Joe Celko's SQL for Smarties is probably the best book for getting 
up to speed.

I'm sure one of you guys knows how to check so that I can avoid:
** Access Error: Port rhm not open
** Where: read-rows
** Near: a: copy db

I think this happens after I have inserted a query and there are 
no results, but in any case I'd like to able to check a port to see 
it is open.

I should add that the code looks like this ...and it works when a 
record is found:
    insert db reduce [s]
	a: copy db

I have a problem when the connection times out and mysql-protocol.r 
wants to reconnect. It looks like it hangs, but on closer inspection 
with trace/net on just gives an infinite amount of low-level reads 
of length 0 bytes.

I can close and reopen the connection, and it works again, but I 
can only do this 5-6 times before it refuses to connect to the mysql 
server. Is this a known problem?

You can try to disengage the auto-reconnect feature and test if the 
connection is still active by yourself (send a ping command for example).

db-port/locals/auto-ping?: off (stop pinging the server and don't 
reconnect automatically).

I found a different solution: manually check if the connection has 
timed out 30 seconds after a query and then reconnect. it's not perfect 
since I have to check at every query, but it works to the extent, 
I've tested it

after the query I store 'now. the next time I do a new query I check 
'now against the stored one. if the difference is more than 30 seconds, 
I close the db connection and open it again immediately

Hello, is there anyone working with Rebol Command and Mysql 4.1.x 
? It appears that the authentication protocol has changed and Rebol/Command 
can not connect to MySQL 4.1.x. But it works fine with MySql 4.0.x.

Ok, but who will develop those modules? Would they be open source? 
Personally, the reason why I love rebol is because it allows me to 
focus on the business, not on the tehnical aspect of a database, 
fastcgi, etc... I am certainly not the one who will develop technical 
modules, simply by lack of knowledge and time. But I really need 
those db access, fastcgi, etc...

It could be RT, or they could come from 3rd parties.  I guess the 
answer is time will tell. You can certainly express your views and 
requests to RT, they may change direction based on customer input. 
However the directions is a bit blurry right now.

Is there any conclusion about DocKimbel's mySQL driver and 4.11 server 
release? Does it work or no? In terms of one project, where we exchange 
data, client agrees to our proposed way, only if we import data to 
their tables on their server. It is Debian linux, mySQL 4.11 or so, 
so before I produce some script and give it to them, I would like 
to ask if the problem of connection/authentication is resolved? Thanks 
a lot ....

I would have to try to download it, install and try it myself, uhm, 
another lost afternoon for testing simple thing :-)

I think you can do that. cronjobs are normal scripts that are carried 
out. they reside in a table of cronjobs which basically is a shell 
script with some time and date information added

thanks, that shoudl do the job. I simply mean - your record structure 
has following fields for e.g. - Name, LastName, Age, Address1, Address2 
and I want to update only e.g. LastName and Address2 fields ....

I have the new MySQL Version 5 Beta installed on my development Notebook 
(WinXP). I was devoloping a Script acessing a current Version 4.1 
of MySQL without any problems and i thought about continuing dev 
work at home. There i ran into an error saying that the autthetification 
protocol of the client is not supported and i should update my client, 
which is the mysql-protocol from DocKimble, when acessing the local 
version 5 Beta of MySQL. Is this a known problem and does anybody 
knows to work around this?? Does anybody know what changed to authenfication 
that is not supported by the protocol??

Re: MySQL Auth.   IIRC  http://dev.mysql.com/doc/mysql/en/old-client.html
I think we used this option "Tell the server to use the older password 
hashing algorithm" and continued without a house call.  If that does 
not work, let me know and on Monday I will ask the programmer who 
did it what did work.

I thought of that before and set the ol_password variable to on. 
But i think i forgot to change the passwords itself. I will try that 
out and let you know if it works. Thanks Mike

I would like to know the result guys, as I am about to create scripts 
for the partner site, and if I fail, bye bye rebol - the trouble 
simply is - they told us - if you want to sync data to us, prepare 
import utilities for us. So - I would still have to do the job, just 
using php or simply something else - which would be a big time loss, 
as I never used it before ....

huh, ok. Finally can test ... while I have set old-passwords in my.ini, 
restarted service, it is not enough - so I installed mysql administrator 
app and changed user password, server probably needs to regenerate 
them to behave correctly ..

Hi Mike, sorry for the late reply. Changing passwords to the old 
algorhitm and setting the variable ol_password to "on" did the job. 
Thanks

has anyone experience with character sets and collations? I can see 
there are variables like character_set_connection() and collation_connection(), 
etc. - can those be set from within the Rebol?

This is related, I think, to my notes about VID and MySQL in the 
View section. In a test that we ran in 2004 we were able to load 
1,000,000 rows in under 30 minutes.  We did not investigate further 
but we thought we could improve this by running parallel loads and 
putting it on a real server instead of a laptop.  This volume was 
equal to the annual volume of  the transactions we were interested 
in so would represent a journal of everything that happened to this 
app as a keyed transaction in one year.   


From that 1,000,000 row database, we were able to create an HTML 
report based on some selected criteria in 2.5 seconds.  

All tests done with REBOL View using Doc's mySQL protocol.

Yesterday afternoon I spent investigating Doc's scheme, and here 
are my conclusions:


- The part of code responsible for password communication with the 
server is in the 'scrambler object. The function whish establishes 
connection with server is 'do-handshake


- rebol's mysql protocol seems to distinguish protocol version 9 
and 10, and I was not able to find out, what does it mean on mysql's 
side of things. It also seems to me, there is long-password item 
already in the stack, but dunno if related, it is just osme constant. 
Protocol version 9 and 10 use different hash functions and different 
crypt functions. I really don't know, where does DocKimbel find out 
how to implement those functions, maybe by looking into mySQL source, 
so I downloaded them


- to read more about passwords in mysql, go here - http://dev.mysql.com/doc/refman/5.0/en/password-hashing.html
. Maybe it is not protocol 9 and 10 related, as you may use old password 
scheme even with new databases. The difference is as follows - starting 
from mySQL 4.11, passwords are stored in 41 bytes wide field, whereas 
with older versions, it was 16bytes. New passwords always start with 
* (asterisk) char, to be easily distinguished ...


- how things might work? Mysql sends scrambled password, and Doc's 
scheme stores it in 'crypt-seed variable.  Data is being sent by 
the protocol by 'send-packet function, where for the password part, 
scrambler/scramble function is used, which calls crypt-v10 or crypt-v9, 
according to protocol versions. Those function use hash-v10(9), floor 
and other functions ...


I think that we somehow need to find out, how to compute hash for 
not 16bytes, but for those 41bytes ...

Now - sorry if I am breaking some licenses, but I will post some 
stuff to my website, and remove it once we are finished:

http://www.rebol.cz/mysql/mysql-protocol.r
http://www.rebol.cz/mysql/password.c

why Doc implemented his hash-V9 and hash-V10 functions this way? 
Don't we have 'sha1 method with 'checksum? (just basing it upon "help 
checksum")

So - I did some homework here, but I am at my limits (well, maybe 
not, but I will be very slow from now on), whereas we have ppl knowing 
C here, and ppl who did some hashing etc. for Beer (Ladislav), so 
guys, if you find some 10 minutes of spare time, please at least 
try to give me some pointers here. As I said - the world is upgrading 
mySQL to 5.0 now, so 4.11 is older, not to mention 4.0.1 or 3.23. 
It is about having mysql free scheme for rebol, or not. And don't 
think every admin will be willing to set old-password parameter for 
his server, as this can be regarded a security risk ...

Actually I don't care what directly is available (as a user), if 
just some things can be done:

e.g. people need to process XML - thus people already knowing XSLT 
and XPATH would like to leverage their knowledge (I asume) - so if 
we get a dialect for this (2.) this is nice, but even nicer if there 
is some mechanism (a generalization) which allows to import an XSLT 
(ast?) or some XPATH query and return the (more rebolesque) according 
Rebol dialect

3. three has always this kind of attitude of being able to do everything 
better in Rebol itself - even if true (?), that's one of the problems 
with Rebol, that outsiders can't afford the time to do many things 
better (themself) or don't care, because they want use some standards 
nevertheless and Rebol drops out as an option


so I vote for 2. with the ability for 1. maybe by the possibilities 
tree rewriting (or dialect rewriting) offers (I have not much glue 
about this - so some of the experts should know)

I agree to MichaelB .... but maybe then I vote for 1, 2 and 3 as 
well, so that 3 can be translated to "standard"

this group exists for a long time, and IIRC initially we were more 
or less discussing rebol - XML interoperability - SAX or DOM parser 
in rebol .... while from what is being discussed now, sounds like 
slightly bit different topic?

Jaime: that's what I meant too. But the discussion jumps around quite 
a bit and as some of these terms are unfamilar (besides the simple 
I know what you're talking about) - it's hard to know what to vote 
for :-).

I guess people have _very_ differing needs in this. Some _really_ 
 need to handle XML with all strings attached to it, and others just 
want to interface to existing technoglogies and read / write xml.

I'd put myself in the second category, if ever I have to work with 
XML again, that is. And I surely hope this won't happen ;-)

I once used XML as a file format, just to play around with it. And 
later I found out, that I'd broken so many rules, that no other gram 
was able to read it anyways. ;-)

and, it's a few lines of code (half a page or so)

I have often missed structural pattern matching in REBOL, something 
like the match statement in Nemerle (I'm sure it's in other functional 
languages but that's the first that came to mind). You could combine 
a structural pattern specification dialect (like XPath) with a structure 
building dialect (like XSLT), and then make the dialect compilable 
to REBOL code that can be used over and over again. It would be like 
a regex compiler for structures - I would use this every day.


All you would have to do to implement actual XPath syntax would be 
to specify a standard mapping of XML semantics in REBOL data structures 
(see my block model in this group from last October) and then have 
compile the XPath syntax (in a string) to the structure matching 
dialect. Then you could work from there.


(Gabriele, sorry if this seems redundant - I'm trying to explain 
tree rewriting in more REBOL-like terms).

You can do some structural pattern matching with parse rules, but 
with how parse is currently implemented it can be a little awkward. 
The lack of arguments to parse rules make recursion quite difficult, 
and the lack of local variables make the rules difficult to use concurrently. 
It is difficult to examine both the data type and the value of elements 
in block parsing, to switch to string parsing mode for string elements, 
to parse lists, hashes or parens, to direct the parse flow based 
on semantic criteria (which is needed to work around any of these 
other problems).


And don't even get me started on the difficulties of structure rebuilding. 
The thing that is the most difficult to do in parse is the easiest 
thing to do with regexes: Search and replace. Didn't we make a web 
site years ago collecting suggestions for improving parse? Wasn't 
a replace operation one of those suggestions? What happened with 
that?


Structural pattern matching and rebuilding currently has to be done 
with a mix of parse and REBOL code that is tricky to write and debug. 
If parse doesn't get improved, I'd rather use a nice declarative 
dialect, preferably with before and after structures, and have the 
dialect processor generate the parse and REBOL code for me. If that 
dialect is powerful enough to be written in itself then we'll really 
be cooking.

Yup. I've posted on rambo about everything I've suggested and more. 
I hope they consider these suggestions for REBOL 3.

my rewrite function works quite well for search and replace. it still 
has the limitations of parse, though, but they don't seem a huge 
problem so far.

Well, I'm back after a long time off the air... Now back online and 
talking from a brand new iMac, and I enjoy it :-)

We'll use it in production very soon. I took a SAX approach because 
we've to manipulate big XML files, and it was a mean to boost performences.

why rebel and not rebol?

(I hope my previous post was read as intended... ironically!)    
;-)  although schema is pretty well defined and actually not too 
badly designed, its just so huge.

I did not do it using parse for lack of knowledge ... and also the 
fact that schema and PARSE do not have the same traversal mechanisms... 
but its still very fast and error report is extremely precise (it 
gives you the full path of the error!)

the change in RebXML was to forego of a few limitiations and to allow 
direct xpath like useage of the loaded xml... which can't be done 
with RebXML out of the box.

Just playing with rebelxml

>> set-xml-data/content 'header/to "[carl-:-rebol-:-com]"
== "<header><to>[carl-:-rebol-:-com]</to></header>"


Now if I start with "header" and "to", how would I get the same result?

How are people creating large xml documents where there are large 
numbers of elements and where the data is being drawn from a database.

Model the document first as a Rebol object!, and then generate the 
xml from the object?

yes, it is him. And his toolset contained several tools IIRC.

Peter - what tools are you referring to? What is xml-to-object and 
parse-xml+?

>> do %xml-parse.r

Script: "A more XML 1.0 compliant set of XML parsing tools." (2-Mar-2005)
** Script Error: Invalid argument:
    Parses XML code and returns a tree of blocks.
    This is a more XML 1.0 compliant parse than the...
** Where: throw-on-error
** Near: func [[{
    Parses XML code and returns a tree of blocks.
    This is a more XML 1.0 compliant parse than the built-in
...
>>

I know we discussed writing a SOAP:// protocol years and years ago 
on the mailing list ... but that never got done.

I read that most web APIs are REST based, but there are still a significant 
number based on SOAP, and also JSON

the only difference I can see between POST and SOAP is that the encoding 
method ... SOAP uses text/xml;

The web and soap/http are in a sense REST applications (REST is just 
WS over HTTP) though both use a limited subset of REST arguments 
and have to work around as such.


The web is limited (by the HTML spec) to the verbs 'get and 'post, 
and post content types of 'application/x-www-form-urlencoded (default) 
or 'multipart/form-data (used to upload files).  For the most part, 
the web is RESTful as we usually only want to 'get resources.  However, 
other operations typically violate REST principles, as all other 
resource actions (create, update, delete) have to squeeze through 
the get/post/url-encode/multipart pipe.  The Rebol HTTP protocol 
as standard is designed to mimic this and requires patching to move 
beyond get/post/url-endode (even for multipart)


SOAP as I understand it, when using HTTP only uses 'post - the post 
content contains the actual request.  Where it varies from the web 
(and Rebol HTTP) is the need for the 'text/xml content type.


REST itself is limited only by HTTP.  It uses theoretically limitless 
HTTP (v1.1) verbs (though most common patterns use 'get, 'put, 'post 
and 'delete).  It uses any encoding.  It uses HTTP headers as parameters 
(eg. the 'Accept header specifies the desired return type).  Therefore, 
any HTTP protocol designed for REST will accomodate SOAP requests.

Probably more a pure HTML question, and showing my complete lack 
of HTML knowledge these days, but how would I go about automatically 
updating a device (an IP Phone in my case) that has a page (http://10.1.1.7/admin.html) 
with a whole bunch of phone numbers in separate fields and a submit 
button that posts changes back? I can generate the page with the 
data I want submitted back, but I can't work out how to mimic pressing 
the submit button from a specific URL.

Almost ... I want to "push" my Address Book (on my iMac) to all the 
IP Phones in our office. I know the address of each IP Phone (10.1.1.x), 
and I can extract the data from Address Book easily enough ... what 
I havn't figured out how to do is replace the manual process of bringing 
up the admin page of each phone (in Safari), keying in the numbers 
and clicking the "Submit" button. Tom's suggestion may be on the 
right track, although I've got no idea what query-string is supposed 
to look like. May be time to dig the old HTML4 manual out! ;)

Web developer extension and FireBug are kind of must-have extensions 
....

ashley, look at the source for the page, and search for the <form> 
tag. if method="GET", as Tom said, look at the url after pressing 
the submit button and just do a read on a similarly composed url. 
if method="POST", you need to look at all the <input> tags, figure 
out what the query string would be, and send it via POST using read/custom. 
(you could also use wireshark or similar to look at the query string 
the browser is sending if you don't want to look for the <input> 
tags)

All works, "read/custom url reduce ['POST query-string]" did the 
trick! Thanks guys.

My little 64 line script now does the following:


 1) Read Address Book vCard file and extract a list of number/name 
 pairs (I prefix the numbers with 'n to assist with lookups)

 2) Read each Linksys SPA942 IP Phone's call history and create a 
 sorted list of number/frequency pairs

 3) Join these 2 lists and create a query string for matches and an 
 exception report for numbers without an address book entry
	4) POST merged and updated name/number pairs back to each phone


Script took 2 hours to write and debug, runs in 2-3 seconds and gives 
us the features of an advanced call management facility for free. 
Once again, REBOL to the rescue (my business partner shook his head 
when he saw this and just said, "but HOW can REBOL do all this???").

Where 'thing is <tag> /attribute or # for text.  And value is [container] 
"text" or none

There must be something, but I don't have anything here that turned 
up, and I don't remember doing one myself. If it helps, you could 
use the JSON converter in %json.r as a starting point.

and you can easily separate attributes from elements, just by affecting 
them to different types.

And you can access it this way:

document/<element>/<subelement>/#attribute
document/<element>/<subelement>/.

My positional version handles multiple subelements of the same type, 
and using strings rather than words lets you use tags that don't 
match word syntax or are case=sensitive.

a later version, using schema validation, understands multiple subelements 
and automatically converts them to blocks IIRC.

so you do document/element/3/subelement/#attribute.

my paths.. namespace works... for sure.  did you know you can have 
colon in word names in R2 !   but i didn;t use that, I just used 
tags directly.  more obvious than strings, and the exact same effort 
and speed.

I was parsing xhtml and other XML of the like. Subelements of mixed 
types in order with text between them than mattered.

And mine is lost :(

my newer version doesn't have the schema validation process.... that 
is a very complex engine to build.   schemas and Parse traversal 
do not follow the same algorythm... so its a bitch to implement.

but the rebxml tools (on rebol.org) as-is are very usefull, so some 
utf-8 support and are less buggy than rebelXML in my previous tests.

yes, the best method is to have some form of dtd or schema, and use 
class-based objects.

class is shared amongst all my-tag instances, and is never bound 
to any of them, so its both very fast and very RAM efficient..  exponentially 
so with large classes.

what's the diff between yours and mine?

my example above, will share the init function amongst all the 2000 
objects... and its going to be faster since no binding occurs... 
well only the CLASS word is bound but its content is the same in 
all instances.

unclear as to why yours is shared and mine isn't

so if I have

pharmacy: make object! [
	address: make addressobj [ ]
]


and the address object has objects in it ... they will be shared??

If that is the case, perhaps I need a 'create function with each 
object, and at init time, iterate thru all the objects calling their 
create function ?