AltME groups: search

sqlab, your answers (in order, and to the best of my knowledge):
- No.

- REBOL_Initialize, REBOL_Status, REBOL_WndProc. There isn't any 
current documentation as to the argument and return types of these 
functions. Some example code would be nice.

- Not to my knowledge, although the question of how to use this dll 
has been asked often. The Plugin group here was formed to request 
that the community create code that uses rebol.dll, and occasionally 
that request is repeated by the person that created the group, but 
progress stalled when people had difficulty calling the dll because 
of lack of documentation.

Thanks! I had recalled him saying that he had posted a link to a 
header in the plugin group, but at the time I had the whole history 
and there was no link. I must have missed it in REBOL2 or something.

I am trying to find out if I can have an application monitor for 
updates and auto-transfer the latest version.  Can an encapped app 
monitor for a new version of the application using the header attributes?

SURFNet detective does this. I don't know how they do it, but when 
I've done this kind of thing, basically, there's a server process 
you can ping to find out the latest version, or just a place you 
can download files and you can check timestamps and sizes, to see 
if there's somethnig new available. 


Also, consider if you need to download the whole app, or if you can 
structure it so the main app is just a kernel, and the parts that 
are likely to change to loadable modules. That's how AltME works, 
and the Detective as well. Then, of course, you want to sign the 
modules to make sure nobody spoofs you into loading malicious code.

This is included:

#include %/c/sdk-2-6-2/source/prot.r

and I see it included prot-ftp.r

What else could be wrong?

is your username and password really user and pass ?

Ok, now I have a related problem. Zone Alarm won't allow my encapped 
program to access the Internet.  I think it is because the program 
is not a view program, and therefore ZA cannot recognize it as a 
program separate from the rebol interpreter. Is there any way around 
this?

No, that is not the reason. Maybe you accidentally denied it access. 
So it should be in ZoneAlarm's list of programs. Go to the list and 
remove your program. Then try running it again. Otherwise, I would 
check if ZoneAlarm has any rules which block ftp. Does another ftp 
client work ?

'radial is default (it seems). I indicate 'linear in the FillType 
object, and it does linear gradient, if that't the only value, I 
use by reference. If I have a second value by reference (FillType/startpos 
in stead of writing 20x20 directly), then gradient change to radial.

Can you use COMPOSE? And does that work around it?

Gregg, yes, if I do a compose as in:

compose [pen none fill-pen (FillType/grad-mode) (FillType/startpos) 
...

then it work correctly. So that is a possible work around. It must 
be a bug, and I'll report it to RAMBO.

It's also worth noticing, that the bug is only there, is the 2 variables 
(grad-mode and startpos) are inside an object. If they're defined 
as stand-alone variables, it works too. So it's a problem with combinations 
of objects and DRAW.

and there's another for mezz.r

Thanks Graham!  I have it up and running now, but it isn't working. 
:-(

Just curious if the sdk package located at http://www.rebol.net/builds/sdk/
requires a newer license.key file?  I already own SDK but this one 
seems to tell me to put my license file in the correct directory 
when I try to run it and I have dropped into every directory I can 
think of.

It was a strange problem I think.  I think I had a pro key and an 
sdk key - that is my only explaination for the problem because Cindy 
sent me a copy of my original SDK download and that key worked on 
the newer one so I think I may have been using the wrong key file.

you can optionally output your prerebol source and run that.  then 
no question of encap being involved.

JeffM wrote in Core:

Can REBOL/SDK define functions that will be called from C and passed 
to a C function? For example:


c-logger: make routine! [ "Define the callback function for error 
tracking."

    logger: [string!] "using string! since a function ptr is just a pointer"
    return: [integer!]
] my-lib "C_Logger"

rebol-logger: func [s] [ print s ]

c-logger :rebol-logger

Be careful though: The current implementation of callbacks in REBOL 
limits you to defining 16 per REBOL process. I'm not sure whether 
it is the making of the routine or the calling of the routine that 
establishes the callback though, so test to be sure. Most scripts 
I've seen use callbacks for logging, like your example, and so only 
call the routine once. Once the callback is established, there shouldn't 
be any limitation to the number of times the C code can call the 
REBOL function, in theory.

The rebol function must complete before the C function tries to call 
it again - or there will be fireworks. Even a short rebol function 
can take 40ms to complete (for example), and if the C function tries 
to call it every 20ms that will stuff up the rebol interpreter.

okay. this is good to know. thanks, guys. something else i couldn't 
find in the documentation was a method of getting a values returned 
by a pointer. for example, void get_value(int field, float *value); 
In this case, "value" is an output parameter. Does REBOL have a method 
of correctly passing a buffer and having it get filled in?

I've been reading the docs both for the preprocessor and for the 
newer include.r script but I'm a bit confused, I wonder if someone 
could help clear it up for me. Here's my situation. I have view and/or 
SDK installed on a PC and a script sitting on my desktop. This script 
needs one or more library scripts which are available from web servers. 
How do i write the start of the script so that I can do all of;

1) Double click and have it run in view

2) Run the preprocessor so that I have an all in one script ready 
to run/copy somewhere

3) Encap it by dragging and driopping it on encmdview.exe or similar


I want to be able to do this without changing a single character 
of the script. I want to access the library files from a web server 
because here at work I often write scripts on various different servers 
and prefer to keep one copy of the libs centrally and not downloaded 
onto each machine.

about include files being on a web server, the official prebol does 
not support that; i think that Ladislav's INCLUDE does, and I have 
a patched prebol.r that does support including from urls too.

it read all filenames in the block and inlines them. after preprocessing 
there are no filenames, so it replaces nothing. without preprocessing 
there is all the files content. in both cases i can do it.

...and if won't be a file! in case the source wasn't preprocessed

and the 'change inserts multiple items. should change position.

when 'include is inbuild it is no problem. but if the main script 
must include to include something and want own content too, would 
help.

i would try a bash-script and do the same as with the windows-shortcut.

Andrew: I haven't published the changed prebol. (I was actually hoping 
for Carl to make it the default in the future.) The problem with 
it is that encmdview would still run with the standard prebol since 
it is built into the exe, so you would still have some problems. 
In the end, it's probably better to use Ladislav's INCLUDE function 
as it gives you more flexibility, and I guess it'll become the default 
too in the future.

** Access Error: Cannot open /c/Documents and Settings/kccadl.KAS/var/website/files/rebol/scripts/libadl.r

hehe, now I know, why rebol/view does not mind #include %../sources/view.r 
line :-)) it just interprets it as issue! and file! :-)

Graham, multiple data streams are how Mac files are stored on Windows 
servers. Also, some antivirus programs use the extra streams for 
storing checksum data, some viruses for storing their payload, the 
system for storing file metadata, and various applications for obscure 
reasons. Any file copy program that ignores multiple data streams 
does so at the user's peril. Since there are few such programs written 
in REBOL (Carl's recent blog post notwithstanding), the lack of data 
stream support in REBOL isn't much of a problem for now.

So, just plain read and write does not touch the alternate data stream

Well, REBOL supports file forks on Mac in a way that you can use 
the same code between Mac and other platforms - the other platforms 
just pretend to have a data fork. However, alternate data streams 
on NTFS are not supported by REBOL so you can get tripped up there.

Well, it's nice that they are semi-supported through a filename hack 
that is actually in the underlying APIs that REBOL calls internally, 
but it is not supported through the actual mechanism REBOL uses for 
supporting that kind of thing on other platforms, and so is not portable.


Try get-modes %file 'forks or the /custom refinement to open, read 
or write with the [fork "forkname"] parameter. On platforms other 
than Mac there is generally only one fork ("data") but if the code 
you write is fork-aware it will be portable. This is the method that 
REBOL should be using to support streams on NTFS or any equivalent 
feature on other OSes' file systems.

Much of the description of NTFS streams and their limitations, as 
expressed by Robert Muench in your link, refer to limitations in 
non-stream-aware code. Windows has APIs that handle streams quite 
nicely AFAIK, including retrieving the names of the streams and deleting 
them if you want to do so without deleting the file or directory 
they are attached to.

How works this filename-hack? And would it make sense to check for 
that for security-reasons for uploads? currently icheck for "is in 
subfolder", can these things add some kind of exe-attribute?

<filename>:<stream>:$<attribute> without the < and > of course. Since 
the default stream is the empty string and all attributes start with 
$ you can skip one of the colons with default attributes. The default 
attribute is $DATA, and what you would normally think of as the contents 
of %file would be in %file::$DATA really.


Executable files aren't marked as such with attributes like they 
are in Unix, but you can hide stuff in alternate streams. It wouldn't 
matter what the file was named if you could pass it to CreateProcess 
or some such, so you can hide executable code in alternate streams 
but without library support you can't call it. You should validate 
the filenames of files you are serving for security anyways, so this 
is just one more thing to look for.

Not really. As it is, there are certain characters that shouldn't 
be in Windows filenames and the colon is one of them. Screening for 
them is a good idea.

REBOL should be stream-aware, period. Preferably using the same fork 
syntax that it uses on Mac and everywhere else.

There is no reason to let that weird NT file-stream-name syntax infect 
REBOL file! syntax if we can avoid it, especially since we already 
have a mechanism for dealing with this kind of thing (and I use the 
term loosely - the underlying semantics are different on every platform).

By the way, the quickest way to determine the characters you need 
to screen for on Windows is to try to change a filename in Windows 
Explorer and type one of those characters. A baloon help message 
will pop up listing the characters that you shouldn't use.

what strategies do people use when you have a few mbs of source, 
and the encap dies due to unmatched brackets ... how to locate the 
offense easily?

load each of the files on each own, using 'load and a loop? 'load 
should fail on unloadable things.

You could load each file until the one with missing bracket is found, 
then copy chunks of text and try to load them in the console repeatedly, 
eg:
	load read clipboard://

Or just keep trying to load the file while commenting smaller and 
smaller chunks of code. That's what I usually do.

if you know that the problem is [, just count up on [ and down on 
] and stop when count < 0. or you can just print only the lines with 
[ or ] in them so you get a "compressed" view of the source and look 
for the missing bracket visually.

The problem actually turned out to be Leo ... and the way I was using 
it.  I commented out a block of code using ; when I should have used 
@, and @c .. so it only commented out the first line of the included 
block :(

I understood the text in link that if you get a lock, you wait for 
a while and try again.. and that by this method even it scales to 
quite big number of concurr processes compared to mysql for example

trying again is already implemented in sqlite.r in the function do-step.
I expanded the wait time with a random duration up to one second.
still not good enough

And now additionaly I connected and disconnected according the link 
.
Again I encountered locks

what disconnection and connecting? :)

He tried only 100 times (by looking at the example he gave) .. and 
I don't know what he means by 500 >>> in line with mysql 

and others- at 500+ simultaneous users you start to get about a 10% 
drop 
because of lock overhead. <<< 


Most probably not 500 processes writing all the time ...  without 
delays

sqlab, use a proxy process that does the priorization. Send requests 
from client to proxy-process and this one handles 1 connection to 
the database.

Maybe my test settings are different from your  requirements.

During testing I let two concurrent processes insert a record with 
three fields, one ip address, a timestamp and one text field with 
variable length from 1k to 8k., and this i do with full speed

I'm pretty sure a proxy process can handle 200req/s at the frontend 
and backend. So if 200req/s is the maximum you can get from one process, 
adding more processes just devides this down. But it won't scale 
up.

If I have some processes serving incoming events and sending their 
data to a central storage process, the central process sometimes 
seems to get an event, but not the data with that event.
When the data really arrives, I do not get the event.
Maybe he got the first event meant for an other connection

I use it for very simple task just so it takes case for locking of 
messages to bots that can come from multiple processes.. but at my 
small/cheap VPS that I use for bots update/insert/delete takes multiple 
seconds which is very bad.. all rebol writes/deletes which it does 
with normal files return imediately on same VPS and yesterday I tried 
rebDB and it was also much much faster for insert/update/delete (almost 
no noticable delay) for the same amount of data (300 lines) as sqlite.. 
funny thing is that sqlite delays the same at these operations if 
there is 300 or 5000 rows in table

and this is no problem with rebol binding as the delays are the same 
if I go to sqlite3 shell

Have you tried put BEGIN transaction and END transaction outer commands? 
Because if not, then it commits each time. The difference is going 
to be ... well, drastic ...

yes, I know for that .. that would come into play if I would be doing 
10 inserts for example ,.. with begin commit it would take just 1/10 
of time as without , but I am doing a simple sql scentence here. 
and I tried begin/commit also with this.

yes, and nothing big.. 5 simple rows

I talked about this in detail a while ago.. now I optimised the problem 
so it's not a biggie any more but I am still mad that all rebol files 
can do changes in moments and when I need to update the sqlite page 
halts for noricable time

Janko - I did small test for you. With no indices the speed was: 
0:00:00.516 and I used LIKE expressions, which need to do searches 
in terms of field values .... 

REBOL []

print "Starting test for Janko ..."

do %sqlite.r

attempt [delete %janko.db]

connect/direct/create %janko.db

sql {
CREATE TABLE [domains] (
[id] INTEGER  NOT NULL PRIMARY KEY,
[domain] VARCHAR  NOT NULL,
[user] VARCHAR  NOT NULL,
[processed] DATETIME  NULL,
[ok_count] INT NOT NULL DEFAULT 0,
[fail_count] INT NOT NULL DEFAULT 0,
[error] TEXT NULL
)

}

sql "BEGIN"

for i 1 1000 1 [

 sql reduce ["insert into domains values(?,?,?,?,?,?,?)" i i i i i 
 i i]
]

sql "COMMIT"

start: now/time/precise


sql {update domains set user = 'u3' where domain like '%1%' and user 
like '%1%'}

print now/time/precise - start

disconnect %janko.db

halt

I will try this on my local computers and then on that VPS.. and 
report you back :)

hm.. I have a question for SQL / sqlite experts : 

I have a query with two JOINS . There is parent table which has 2 
subtables ... in each subtable I need to aggregate (SUM) some value 
... 


select i.*, SUM(ib.price * ib.qty) as amount, SUM(ip.amount) as payed_amount
	from invoice_sent as i 

  left outer join invoice_sent_b as ib on i.id = ib.id_invoice_sent

  left outer join invoice_sent_p as ip on i.id = ip.id_invoice_sent
	group by i.id order by i.title;


The problem is tha because of join , the amount is correct , is the 
sum of values in invoice_sent_b , but payed_amount is multiplied 
by the number of rows invoice_sent_b has . 


I understand why this happens, but I don't know how to prevent it, 
if I want to get all data in one query.


( I know how to solve the prolem , but it's outside the DB so it's 
more of  a hack  -- I also ger COUNT of _b table rows and divide 
second SUM by it on client )

Say, where there are a moderate number of moderate sized scripts 
- could it be quicker just to fire up an SQLite connection and access 
the scripts/modules from records as opposed to flat file?

If you had persistent connection and if sqlite does some caching 
it could be faster, but if you need to open a connection on each 
request I think it would be much slower because that is more expensive 
(I assume).. it probably also matters how many scripts do you need 
to run per request

Check if you have the right path to file.. and make it an absolute 
path.. and if you have the right version .. sqlite != sqlite3

I apologize if this is a repeat of an earlier message - I thought 
I submitted, but never saw it appear.


Question: With REBOL/View 2.7.6 under Windows XP, using SQLite driver 
v1.0.6, should I be able to use the concatenate operator (||) in 
a SELECT statement?  Query works under the sqlite3 command line, 
and a similar SELECT without the concatenate returns values as expected:

>> sql "select jobid, role from roles where userid = 'cda6'"
== [[1124 prgr] [1125 test]]

>> sql "select jobid || role from roles where userid = 'cda6'"
** Syntax Error: Invalid integer -- 1124prgr
** Near: (line 1) 1124prgr
>>

Jack, finally I saw the problem


The return of sql isn't two blocks of string, it's two blocks of 
word. So 1124prgr isn,t a valid word and isn't a valid integer

>> a: sql "select jobid, role from roles where userid = 'cda6'"
>> type? first first a
==word


You need to add the /direct refinement to sql and it returns strings

>> sql/direct "select jobid || role from roles where userid = 'cda6'" 
==[["1124prgr"]["1125 test"]]

Robert - thanks...that's what I eventually did.


Manu - thank you for making clear what the /direct refinement does...I 
had read about that in the SQLite Driver Guide, didn't understand 
it since REBOL is still new to me, and forgot about it.  Works perfectly! 
 So much to learn....

stupid me, I only now realized sqlite driver supports prepared statements 
too and I was concatenating strings all along.

The sorting problem with collations applies to Unicode, not just 
UTF-whatever, so it is a problem. Mostly a problem of size: The actual 
colation sorting code is small, but the collation data is large. 
Add more than one or two and REBOL gets huge.

A solution would be to move the sorting out into the host code, where 
it can be made to use any system-specific sorting code, which should 
work for platforms with good Unicode support like Windows, OS X and 
some Linux distros. The problem with that is that the data would 
probably need to be migrated to the host format before the host routines 
could be used; string! is not UTF-8 or UTF-16 internally, it is UCS-8 
(in theory), UCS-16 or UCS-32 (in theory), all fixed-length encodings. 
That would add a lot of overhead to sorting.

you can wrap the fields in a panel and use:

set-face my-panel [db/:counter/2 db/:counter/3 ...]

And, I have abstracted this idea in that a function walks over the 
set-words of an object and use these as columnnames.

There is no standard I know of for mapping data to faces. A number 
of us have rolled our own systems over time, each with our own critieria 
and design aesthetic.


As a simple starting point, consider setting up declarative mappings 
and driving a data-exchange loop. e.g.

face-field-map: [
    lname 2 fname 3 spouse 4 
    email 5 hphone 6 cphone 7 
    addr 8 city 9 state 10 zip 11
]

foreach [face-name field-index] face-field-map [
    set-face get face-name  pick db/:counter field-index
]

--

You can use rebsqlite with core on windows (it allows dll inclusion) 
but how can you use it on linux? It seems linux rebol doesn't allow 
.so . I used rebpro which does this but now I also need to send an 
email from it and it gives me some strange message "Set-Net not provided."?


Maybe I am doing something wrong/using the wrong tools... how can 
you use rebol with sqlite on linux??

I don't see it mentioned anywhere that this license / upgrade is 
multiplatform. The select box is the same as before where you choose 
the platform. I have no problem giving $50, but I am running on fumes 
here and $350 shows on my account, especially if I am don't need 
it really, because rebol core/view on windows work with rebol sqlite 
and I haven't seen anywhere that linux version has different logic.

hm .. maybe access to dll/so is available only to view and not core. 
and because I use core (cmdline) on linux server it doesn't work

Yeah, for now /Core doesn't have /Library. /Pro does, and can be 
encapped to resemble /Core with /Library; that is the only difference 
between /Pro and /Base.

Yes, /View/Pro is now free and called /View.

aha, I tried is. it's not that windows supports dll-s and linux not 
but the view does and core doesn't. I guess I am gonna have to buy 
the linux sdk also.

Brian: huh.. I see now I made typo when making file that do-es all 
mezz and proto files .. if I load it correctly set-net works!

Yup. And (on Windows at least) set-net is by default that "Set-Net 
not provided." function in /Base, /Pro and /Face, on purpose. The 
only reason that function is in the encap bases at all is because 
it is called in the startup code of /Face.

Rebol and DSL's are hand and glove. Is there a SQL dsl anywhere? 
I found the driver from dobeash. Is there more?

I'm not sure if that would be worth the effort, SQL can get fairly 
complex and if a REBOL SQL wouldn't implement everything (even specific 
databases prorietary extensions to SQL) that wouldn't be a noticable 
gain in simplicity. Wouldn't it still just be SQL written in REBOL's 
syntax? Actually, I think it's more like SQL already is a perfect 
example of a DSL on it's own (not implying here that SQL is perfect, 
just the example is).

And this is a snippet:

do %sql-protocol.r
    db: open sql:my-db
    insert db {
        SELECT * FROM a, b
        WHERE a.c2 = b.c1 AND a.c1 = 1
        ORDER BY 1, 2 DESC
    }
    foreach item copy db [probe item]
    close db

Isn't this better than working with strings?

I have a REBOL/View application where I am doing SQL calls in a face 
timer to update the face. I am having an intermittent issue where 
the result of one face's SQL statement is showing up in another face. 
I am doing a copy of the result set based on what I've read in the 
Dobeash docs.


It appears that the face timer code blocks and another timer event 
occurs and does a SQL call. Does anyone know how to make a timer 
function finish before another one triggers? I may also be using 
the driver incorrectly.

For now, I've wrapped the SQL calls with "busy" flag logic and that 
seems to work -- but it is a hack. The fact that a SQL function call 
inside a face timer event returns a dataset from a different face 
is shocking to me. REBOL should handle this properly in the background. 
My mental model of single-threaded event handling (rooted in classic 
VB) leads me to believe that this is a bug.


In addition to returning data to the wrong face, the SQLite shared 
object will also sometimes seg fault. It working fine now, though, 
with my busy flag hack.

Anyway I use asynchronous database queries and have never seen this 
issue.

Graham, yes, I am triggering queries using timer events, and the 
sql results are being returned to the wrong calling function. My 
classic VB way of doing things doesn't work. How exactly are you 
doing your queries? Putting them in a queue to be serviced by a single 
function? For me that would also require including the face, so it 
can be updated when the sql call completes.

Here is Gabriele's response to my bug report (RAMBO ticket # 4411)


This is not really a REBOL bug. The SQLite driver is doing a WAIT 
1 in the DO-STEP function; calling WAIT inside an event handler (such 
as a face's timer event) is not really supported by REBOL and should 
be avoided. If you can't avoid it, you must be prepared to handle 
other events happening during that WAIT call, and you need to be 
very careful with blocking requesters (SHOW-POPUP, INFORM etc.) as 
they're likely to mess things up (they call WAIT as well).

My suggestions are:


1) File a bug report with the SQLite driver. There needs to be a 
way to avoid the WAIT, though I guess this is going to be complicated 
in R2.


2) Disable other events while you're using the SQLite driver, eg. 
clear SYSTEM/PORTS/WAIT-LIST and restore it afterwards.

3) Use a "global" flag like you're doing.

-Gabriele

most likely, you'll only have the event port in the wait list, so 
it's just a matter of removing it and adding it back later. I can't 
guarantee this works 100% because I haven't tested it.

another alternative could be:


4) don't use face timers, but rather replace do-events with something 
like:

    forever [
        wait 1
        do-your-sql-calls
    ]


You'll still have other events happening while you're doing your 
sql calls in this case though, not sure if that can be a problem. 
The cleanest solution would be to change the driver so it does not 
wait, but I don't know SQLite so I can't say how easy that would 
be.

Thanks everyone for your explanations and suggestions.

and has a driver for sqlite I think

Is there a way to force sqlite to release a database. 


I get a currupt db now and again and I want to automate the process 
of re copying an older version over it once I detect it is currupted 
using a try block.


But once the database is accessed it won't let me delete it or disconnect 
from it. Anywayaround this?

And has anyone been experienceing curruption to their db's...looks 
like binary characters get written into some fields randomly. Other 
people do not seem to be having this problem with the program on 
their computers.

xp and vista. I just had the issue on vista. I do not remember if 
it has occured on my xp machine.

Why Graham? desktop as opposed to server?

I'm referring to the SQLite version. The current is 3.7.5. You can 
check the synchronous setting by executing the following query -- 
pragma synchronous. It will return a 0 (off), 1(normal) or 2 (full). 
See http://sqlite.org/pragma.html#pragma_synchronous.


I've had database corruption before because I didn't backup the files 
correctly. It's best to use the backup API and not just copy the 
files.