Handling very large databases
[1/4] from: chris::starforge::co::uk at: 19-May-2001 10:25
Hi,
I'm probably missing something obvious here, but anyway: say I have a
file containing a meg or so worth of
[string! date! integer! string! ...] [string! date! integer! string! ...] ...
and I need to display blocks 120 to 135. Loading the whole file into memory
just for a few blocks in the middle of the file is obviously a waste of
time and resources. As this is a cgi script, and my hoster gets a but
steamed if scripts lock up the system for too long, I would like to optimise
this case so that I can just load the blocks I need, say something like
projectdata: loadrecords/start/end project-datafile 120 135
Is there a way I can do this?
Chris
--
New sig in the works
Explorer 2260, Designer and Coder
http://www.starforge.co.uk
--
Things I'd Do If I Ever Became An Evil Overlord
9. I will not include a self-destruct mechanism unless absolutely
necessary. If it is necessary, it will not be a large red button
labeled ``Danger: Do Not Push''. The big red button marked
``Do Not Push'' will instead trigger a spray of bullets on anyone
stupid enough to disregard it.
Similarly, the ON/OFF switch will not clearly be labeled as such.
[2/4] from: pa:russo:perd at: 19-May-2001 19:21
>Hi,
> I'm probably missing something obvious here, but anyway: say I have a
<<quoted lines omitted: 8>>
>Is there a way I can do this?
>Chris
Hi Chris,
You can find some useful tips on "REBOL/Core User Guide 2.3".
This is the beginning of the "Ports/Other Port Modes/Direct Port
Access" section:
...The /direct refinement opens an unbuffered port. This is useful
to access files a
portion at a time, such as when a file is too large to be held in memory....
.
An example follows.
You'll find only sketchy informations, but it's a start.
HTH
--
Paolo Russo
[pa--russo--perd--com]
_________________
PERD s.r.l.
Virtual Technologies for Real Solutions
http://www.perd.com
[3/4] from: chris:starforge at: 19-May-2001 18:52
#19-May-01# Message from *Paolo Russo*:
Hi Paolo,
> You can find some useful tips on "REBOL/Core User Guide 2.3".
Heh, I should have RTFM ;)
> This is the beginning of the "Ports/Other Port Modes/Direct Port
> Access" section:
> "...The /direct refinement opens an unbuffered port. This is useful to
> access files a
> portion at a time, such as when a file is too large to be held in
> memory....".
Hmm.. sounds like this could be a lot of fiddly work. Maybe for version 2.
My files shouldn't get to unmanageable for a while anyway.. Thanks for
the tip though.
Chris
--
New sig in the works
Explorer 2260, Designer and Coder
http://www.starforge.co.uk
--
The Scripture of the Master Programmer [Principals 1:60]
Never give a simple, straightforward answer when a long and complex one
will do.
[4/4] from: dmurrill:mindspring at: 19-May-2001 20:03
Hi Chris,
Is it possible to write a CGI script (to act as a function) to copy/parse
blocks 120 to 135 of the file, dynamiclly bind it to a new (made on the
fly)
script from the same CGI and return that to the window/browser/app?
This could be your answere, maybe? I don't know, but i do this without CGI
for email that's dumped into files/forms that are allready downloaded in
my web-browser.
----- Original Message -----
Notes
- Quoted lines have been omitted from some messages.
View the message alone to see the lines that have been omitted