World: r3wp

Terry;  Sunanda's contest was using  the parse technique on some 
very large

indexes...for skimp.  A foreach solution was actually the fastest 
for a teenie window of

time, until Romano posted the parse winner.  But...it was a different 
problem set.

It was a very informative contest in terms of efficient REBOL coding.

T: Yes, that's my last function (after I reread your post)

These don't feel right. .. looking for the equiv. of  SQL's  "select 
value where key = 'one'" .. Isn't rifling through a 100mb hash table 
using parse similar to rifling through an un-indexed SQL table?

The rse-ids.r  file seems what Im looking for .. need to have a play.

Yep.  You could sort, find first, find last and copy the range?  
But that introduces sort...

There is a blog entry about hash! but we have to wait till R3.  RSN.


Yeah, there was some high-level optimizing going on for that res-ids 
little beauty.  :)

; one more :)
select-all: func [block [any-block!] key /locals result val][
    result: copy []

    parse block [any [to key skip set val skip (append result val)]]
    result
]

That I like... :)  And Terry;  You may be surprised at the timings 
of that  to  sequence.

(thru key) would work as well, if not better, than (to key skip) 
 :)

Or  thru  sequence  :)

What do you use for the key? a word value .. 'one ?

Yes -- select-all data 'one

doesn't work

sorry, does work..

>> n: make hash! [one "test1"  two "test2"  one "test3"]
== make hash! [one "test1" two "test2" one "test3"]
>> select-all n 'one
== ["test1" "test3"]

(I've been doing so much javascript and php  lately, im really starting 
to lose whatever rebol understanding I once had)

Ak, js is passable, php? :)

This sort of hash table won't work anyway due to the 'word limitations 
of the current core

It will work with other keys, but has the same issue as 'select in 
that values can be mistaken for keys.

exactly  (or as the French say... exact)

Im tryin real hard to get my simple data into a Rebol hash table, 
or blocks.. whatever.. but it seems like traditional Relational DB 
is the way to go.. even used only as a flat file DB :(

; This may slow things down a little:

select-all: func [block [any-block!] key /locals result val fork][
    result: copy []
    parse block [
        any [
            thru key fork:

            (fork: pick [[set val skip (append result val)][]] even? index? fork)
            fork
        ]
    ]
    result
]

I'm a little confused... what word limit are you bumping into?  I 
thought the limit was

only for set-words?  Can't blocks have any number of symbols inside? 
 Until ram
is gone...

I guess the alternative is *sigh* waiting for R3 where these issues 
will be addressed...

terry is ordering your data when you insert it prohivitive?

sigh alright

somewhat tom

knowing more about how it will be accesses inser delete moce selecr 
would help choose a stratagey

if everytime you do a 'write' you need to sort 400mb file.. i would 
say yeah

Ok.. it's like this..

sorted keys you could binary seatch  to avoid linear scan

Terry;  Go back to the QRAM days, segment everything. :)  It only 
adds an order of
magnitude in complexity.  :)  Love Wintel...

Sorry.  I shut up now.

My data is structured using a semantic network model..  entites.. 
ie:

tweety   isa   canary
canary   isa  bird

bird   haspart   feathers
bird   haspart  beak

so you have seperate hashees of different relationship types

so currently its stored as entity - attribute - value  in a single 
db table using 3 cols

isa hash and partof hash

(lots of arguments out there regarding EAV as DB model...  most say 
it cant be done.. I say rubish.. works beautifully on a university 
project currently at 3 million rows of data)

it does work very well

It just seems that relation db is overkill   all i need is key/values 
 ... something like this.. 


n: [ tweety [ isa  "canary"  hascolor  "yellow"] canary [ isa "bird" 
haspart "beak"]]

I am wondering if you have a smallish number of relationship types 
 if splitting  into EA tables  getes you anything

the attributes are smallish

Brian, I believe words in blocks are limited as well

I do REBOL with pretty small datasets.  Could be  I just did a test 
of 1'020'000, but
I was inserting the same word with a number...not unique words.

yeah.. unique is the key  .. i need unique

Is there a need to maintain three columns, over say -- (bird [haspart 
feathers]) or (bird haspart/feathers)

(moving to DB Chat)

Brian, try -- repeat x 10000 [unless attempt [to-word join "w" x][break/return 
x]]

Warning: this will inhibit your ability to make new words in your 
console session.

any benchmark docs comparing certain functions with others?  ie: 
  foreach [ a b c ]  n..  vs   forskip n 3   ?