[REBOL] Re: possible data format ... Re: Re: dbms3.r 01
From: rgaither:triad:rr at: 15-Jan-2002 13:47
>>A good point. I might lean towards a single file for each kind of
>>BLOB (or should that be TLOB :-)). I still have a strong aversion to
>>creating lots of files to represent the DB.
>Why? :-) We really do use one file strategy for database table, one file
>for index bag (multiple orders), one file for memo. I think that we are
It is really only a preference. I am used to systems with 100s of tables
and everything included in 1 db file. :-) I remember the dBase way of
doing things as a pain, but admit in this case it probably is not worth
the effort for single file or even reduced file versions.
>talking lightweight db here anyway, so, why are we talking blobs here?
I think we need to talk about large text fields at least. I would love
this little db to manage my saved mail in a more usable format for
>:-) In reality, I prefer each table in separate file, it is also somehow
>safier to me and you can copy tables around your harddrive to process
>them in external analatic tools. Once I use ODBC driver for e.g., it
>treats directory containing multiple db, index, memo files as one real
>database, where you can query its tables.
Good points all.
>As for putting changed records at the end of file - I thought that
>constant size fields/record would be enough? That way you could just
>perform 'find on the file (seeking for record you are about to change),
>and simply replace it. Longer text sizes should come to external memo
I am not suggesting a fixed length record or field where this is possible.
>file, but first - I am duplicating XBase functionality here and 2) I can
>feel that is something which you probably would like to avoid to happen
>for "simple" Rebol dbms system :-)
Yes duplication with any rdbms is happening with this effort. It is part
of the problem though that all those products are external to REBOL
and thus limited in their own ways.
>.. so, btw - I am somehow lost - what do we want to achieve? Any
>conclusions so far? :-) We can try to prepare description for several
>scenarios provided, and continue to post their pros/cons and then decide
>upon implementation. ...
I apologise for this. :-) I want to make an effort to bring focus back to the
project before I distract the discussion any further.
I think the next step is to settle on a "starting" file format so we can
get back to the functional questions Gabriele proposed. :-)
What I (my opinions only) conclude so far, though review of some of
the existing implementations may change this -
1. Text files for persistent storage
2. REBOL Blocks for grouping values
3. Native REBOL representation for values
4. Use single base directory - no directory structure required
5. Use multiple (named) text files to map to tables, indexes and large text blocks
6. Keep at least the table data file simple, readable, and self contained
7. One record per line
8. Use some manner of file append to manage record changes
9. Delete record operation is only a marker until packed
10. Need utilities to pack db and provide canonical form
11. Every row has an automatic internal "row-id"
12. Manipulate records as blocks of values
13. Result sets are built as blocks of blocks of values
14. The in-memory format should not be constrained by the persistent format
The rest, and yes there were more good points only not quite as
well defined, can wait for the next round of design theory. :-)
Oak Ridge, NC - USA