KDB was {Re: Re: dbms3.r 0}
[1/5] from: jason::cunliffe::verizon::net at: 15-Jan-2002 1:24
http://www.kx.com/developers_faq_kdb.htm
<quote>
Why is Kdb so fast?
The main reason is that Kdb tables are "inverted", i.e. the data in each
column is stored together, instead of in the row orientation used by most
RDBMS's. The column data is also organized for optimal processing speed.
</quote>
./Jason
[2/5] from: rgaither:triad:rr at: 15-Jan-2002 12:17
Hi Jason,
Thanks for the links!
I'm always looking for new DB technologies. :-)
>http://www.kx.com/developers_faq_kdb.htm
><quote>
<<quoted lines omitted: 3>>
>RDBMS's. The column data is also organized for optimal processing speed.
></quote>
I remember some small personal information manager (pims) databases
being "Fully Inverted" - which I believe means every column is
automatically indexed. When the db engine is optimized for this
kind of indexing and combining result sets it can be incredibly fast.
This speed usually comes at the cost of extreemly large storage
representation though and only in the read only search operations.
The same things that make search and analysis so fast cause
updates to be very slow - needing to keep all that indexing and
duplicate data in sync. This is why it was used in pims where the
read/write ratio was heavily skewed to reading. Some of the same
techniques are applied in reporting "data cube" implementations
as well. These techniques don't work as well with online transaction
processing (OLTP) systems like those typically used in business
applications.
Thanks, Rod.
Rod Gaither
Oak Ridge, NC - USA
[rgaither--triad--rr--com]
[3/5] from: jason:cunliffe:verizon at: 15-Jan-2002 14:36
Hi Rod
> Thanks for the links!
>
> I'm always looking for new DB technologies. :-)
Good.. Then you might appreciate this gem I found browsing through the Kdb
list archives last night:
<quote>
- ----- Original Message -----
From: "Andrei Moutchkine" <[muchandr--CSUA--Berkeley--EDU]>
To: <[kdb--listbox--com]>
Sent: Sunday, August 27, 2000 2:13 PM
Subject: Re: http://www.kx.com/products/index.html
> The nihilistic
> tendencies notwithstanding, K and kdb are actually ueber-OO in this
> respect - they are all about re-applying the powerful aggregation trick
> over and over across all granularity levels of your data while exploiting
> all that homogeny present at each level.
right, right. i haven't seen it put this way before, and i think
it captures what's going on.
> Your typical relational database
> doesn't fare so well. In a typical translation between the language of
<<quoted lines omitted: 6>>
> the more random the data, the less need there tends to be in persistent
> storage thereof :)
this is a good insight. thanks.
</quote>
./Jason
[4/5] from: g:santilli:tiscalinet:it at: 17-Jan-2002 21:31
Hello Jason!
On 15-Gen-02, you wrote:
JC> <quote>
JC> Why is Kdb so fast?
JC> The main reason is that Kdb tables are "inverted", i.e. the
JC> data in each column is stored together, instead of in the row
JC> orientation used by most RDBMS's. The column data is also
JC> organized for optimal processing speed. </quote>
Well, that's like having all columns indexed. It's nice when
you'll be searching an all columns with the same frequencies, I'm
not sure in other cases (in paticular, inserts and updates will be
much slower).
Regards,
Gabriele.
--
Gabriele Santilli <[giesse--writeme--com]> - Amigan - REBOL programmer
Amiga Group Italia sez. L'Aquila -- http://www.amyresource.it/AGI/
[5/5] from: jason:cunliffe:verizon at: 18-Jan-2002 13:31
> JC> <quote>
> JC> Why is Kdb so fast?
<<quoted lines omitted: 6>>
> not sure in other cases (in paticular, inserts and updates will be
> much slower).
I don't know about that. Kdb is incredibly fast [and small] and scales well.
It really is worth downloading the demo and taking it for a testdrive,
especially with its built in http port. Like I said, its very rebolistic. I
am not proposing we use it directly, but yes for hands.on inspiration.
The desing aspect which I think is most relevant to your work, is the way
Dkb is built on top of the K data analysis language. This is what strikes me
as rebolistic.
Your preference for REBOL base structure is entirely in keeping with this.
Looking over the K and Kdb docs and playing around with the demo
interactively certainly encourages one to think differntly about the rdb
dialect set which might be possible and desirable. That may be puting the
cart before the horse, or it might be a viable alternative.
regards
./Jason
PS. I proposed to install a vanilla site to help rdbms3r. Have been
rebuilding our server but ran into nasty scsi motherboard problem.
Mercifully now sleuthed and fixed, so hope to have be back online next week.
I'll be able to focus on using it then, instead of building it :)
Notes
- Quoted lines have been omitted from some messages.
View the message alone to see the lines that have been omitted