World: r3wp

forinstance, in Zope, I have a single sql connection which is used 
by all the "cgi" requests.

But if you want to access a mysql server, the cgi script loads each 
time, right?

loads the mysql protocol, that is.

depends on how it is written.

can you give me an example of a cgi script that access a db, and 
doesn't load the protocol each time?

any Zope cgi script that access a database does that.

I'm referring to Rebol

if I have a rebol daemon running that has a connection to a odbc 
database, then my rebol cgi script an interact with that.

via what, tcp?

yes

Well, I wonder what has less overhead.. reading a 23kb protocol with 
each call, or taking the time to do a tcp call?

clearly the latter

my portal does this .. keeps an open sql connection which is used 
by incoming requests

what do you use, rugby?

yes.

What is faster, reading a 100 byte text file, or accessing 100 bytes 
from a DB?

you would have to do timing studies on questions like this ...

Yeah, just doing that now.

after a text file being read from a directory is another db operation.

after all ..

the filing system is a database

is it?

Ok, doing some timing tests.. using 10000 loops... 


Firstly, the mysql is choking around the 9800 -9850 mark by denying 
access at that point...

(i think my test code is messed up)

Actually, i was counting down, and mySQL is choking after 150 calls 
or so.

are you waiting for responses before you call it again?

Forced to reduce the number of iterations to 100 for mysql sake.. 
and even then it would choke 50% of the time.. results.. 


Reading the DB used mysql-protocol.r with join mysql://.. localhost 
{select email from table where name='name' }
Reading the file used a: load %file.txt, b: select a 'email
mysql - 1.35 seconds (using time/now/precise

file - 0 seconds (timestart - 18:35:25.531, timefinish - 18:35:25.531)


Conclusion.. no contest.. loading and selecting from file wins hands 
down.

Graham, if i need to wait for a response, then i would need to add 
that time to the total for mysql.

and if you have 1million files in the local directory?

good question... and one million files in the DB as well.

10000 iterations using the load file method.. 0:00:00.781

I doubt very much that the DB could do that.

and then later on if you need to copy the 1million files from the 
filing system, or copy the mysql database .. which is faster?

another good point.. think ill load up a directory with a few 100,000 
files and see what happens ;)

If you dont' here from me, you'll know that I mashed my drive :)

hear, that is

well, copying the messages from ios took absolutely ages ...

10,000 iterations .. folder has 100,000 27byte files  - 0:00:00.765 
seconds

Took awhile to create 100,000 27 byte files.. long after the rebol 
script had finished, the os was still handling the cache...
Size: 2.57mb
Size on Disk: 390 mb

but we needed 28 byte files to test this out!

I think the results speak volumes.

and how much space does the db require to store the same data?

don't know

Does anyone know how to setup a subject filter in gmail?  Can you 
do wild cards like *[Rebol]* for subject filters?

size of DB approx 36 mb.

Took about 6 mins to move the directory.   So it appears to be a 
trade off.
Benefits of using files... 

- much more flexible (ie: a 10mb binary file in a DB is not a good 
idea)
- easier to manipulate.
- magnitude faster access time

the magnitude faster time breaks down pretty quickly when you need 
to retrive a file based on it contents not its title

No, my experiments were using the file AND it's contents via SELECT

using indexes in your database?

primary key?