World: r3wp

results with this function fixed are similar...

preparing sparse data
6000005
sparse tests:
feach(): 5.4.3.2.1. -> 0:00:04.89   4 matches found
----------------
find-fast(): 5.4.3.2.1. -> 0:00:00.719   4 matches found
----------------
find-really-fast(): 5.4.3.2.1. -> 0:00:00.25   4 matches found
----------------

dense match with key/value collisions
6000000
feach(): 5.4.3.2.1. -> 0:00:13.328   221606 matches found
----------------
find-fast(): 5.4.3.2.1. -> 0:00:14.843   221606 matches found
----------------

find-really-fast(): 5.4.3.2.1. -> 0:00:06.5   221606 matches found
----------------

dense match without key/value collisions
6000000
feach(): 5.4.3.2.1. -> 0:00:13.843   222405 matches found
----------------
find-fast(): 5.4.3.2.1. -> 0:00:09.672   222405 matches found
----------------

find-really-fast(): 5.4.3.2.1. -> 0:00:06.375   222405 matches found
----------------
done

Thanks Maxim

just trying to grok it :)  so 2,000,000 keys against 6, 000,000 values?

setup as records of 3 items,  1 key, 2 values.

3 items  (1 key, 2 values).

2 values?

not following ( as usual)

a typical hash (for example is [key1 value1 key2 value2]  .. one 
key, one value

in the code... 


rsize: 3   is the record size...  like the /skip value in most series 
handlers  

my two funcs will adapt, since you provide it the record size


but ... ehehe, I just realized that find HAS the /skip attribute... 
doh!!! the above can be made MUCH faster still, especially by removing 
the need for the keys (which take a bit of time to generate on large 
lists).

my ultimate goal is storing triples..  [a1 a2 a3 b1 b2 b3...] preferably 
strings if that's fast enough

where i can search against any part of the triple , and return the 
whole thing (triple)

i'll only be generating the initial series rarely, but appending 
/changing/ removing often

ok. doable, with almost the same speed as the find-very-fast   func, 
but without the need for its keys argument.

here's a better example  

dataset: [ "subject" "predicate" "value" "bob" "fname" "Bob" "bob" 
"age" "42" "Maxim" "age" "unknown" ... ]
so the triple is subject, predicate value  

with the ability to say quickly return   every triple  where "predicate" 
(the second part of the triple)  = "age"


i'm doing this now with MySQL as a triple store, but series should 
be much faster

I've got it done.... just testing it right now.

if strings are quick enough and not much overhead, it's possible 
to use key/value   but the key is made up of subject:predicate ie: 
"maxim:age" "unknown"  so that select/part key ":age" would return 
everything including "unknown"

using find/skip and an index for what item to search for in the record... 
we get by FAR the fastest search... if you count the fact that generating 
the keys for find-very-fast often takes as long as the search itself, 
in dense searches.

ultimate-find below gives the results of this new function


preparing sparse data
6000005
sparse tests:
feach(): 5.4.3.2.1. -> 0:00:04.89   4 matches found
----------------
find-fast(): 5.4.3.2.1. -> 0:00:00.719   4 matches found
----------------
find-really-fast(): 5.4.3.2.1. -> 0:00:00.234   4 matches found
----------------
ultimate find(): 5.4.3.2.1. -> 0:00:00.594   4 matches found
----------------

dense match with key/value collisions
6000000
feach(): 5.4.3.2.1. -> 0:00:13.343   221606 matches found
----------------
find-fast(): 5.4.3.2.1. -> 0:00:14.813   221606 matches found
----------------

find-really-fast(): 5.4.3.2.1. -> 0:00:07.718   221606 matches found
----------------
ultimate find(): 5.4.3.2.1. -> 0:00:06.735   221606 matches found
----------------

dense match without key/value collisions
6000000
feach(): 5.4.3.2.1. -> 0:00:13.969   222405 matches found
----------------
find-fast(): 5.4.3.2.1. -> 0:00:09.812   222405 matches found
----------------

find-really-fast(): 5.4.3.2.1. -> 0:00:07.672   222405 matches found
----------------
ultimate find(): 5.4.3.2.1. -> 0:00:06.531   222405 matches found
----------------
done

next will be ultra-ultimate XD

ultimate-find: func [
	series
	value

 index "field you want to search on, should be (1 <= index <= record-length)"
	record-length
	i "iterations"
	/local s st result
][
	prin "ultimate find(): "
	st: now/precise
	while [i > 0][
		prin i
		prin "."
		result: clear [] ;length? series
		s: at series index
		until [
			not all [
				s: find/skip s value record-length

    insert tail result copy/part skip s (-1 * index + 1) record-length
				s: skip s 3
			]
		]
		i: i - 1
	]
	
	prin join " -> " difference now/precise st
	print ["  " (length? result) / record-length "matches found"]
	
	head result
]

searching for strings will be slower... probably much slower... just 
try it out with some data  :-D

what does it mean "field you want to search on"?

what item of your record you want to match against... basically what 
you meant by searching  subject, predicate or value

yeah, so if i say 1 then that's all subjects?

yep it will match the value against subjects only.

no go then

>> dataset

== [6 27744 92191 1 61175 9905 9 62225 72852 7 31935 71556 4 59248

>> ultimate-find dataset 1 1 zz 1
ultimate find(): 1. -> 0:00:00.031   0 matches found
== []

should have picked up that fourth index (1)

this worked.. 
>> ultimate-find dataset 6 1 zz 1
ultimate find(): 1. -> 0:00:00.219   1 matches found

zz should be 3.

oh.. i thought zz was the length? of dataset

ah.. that works GREAT

>> dataset: [1 "a" "B" 4 "h" "V" 1 "z" "Z" 4 "p" "d" 4 "k" "i" 4 
"y" "o"]
== [1 "a" "B" 4 "h" "V" 1 "z" "Z" 4 "p" "d" 4 "k" "i" 4 "y" "o"]
>> ultimate-find dataset 4 1 3 1
ultimate find(): 1. -> 0:00   4 matches found
== [4 "h" "V" 4 "p" "d" 4 "k" "i" 4 "y" "o"]

very nice

ultimate find(): 1. -> 0:00   1 matches found
== [1 "a" "B"]

:-)

oops missing cmd line...

so if the dataset is key/value just use 2 as the record-length

>> ultimate-find dataset "a" 2 3 1
ultimate find(): 1. -> 0:00   1 matches found
== [1 "a" "B"]

yep

cool

i inserted "maxim" "age" "unknown" and appended "terry" "age" "42" 
into the dataset containing 6 million records.. 

>> ultimate-find dataset "age"  2 3 1
ultimate find(): 1. -> 0:00:00.093   2 matches found
== ["maximn" "age" "unknown" "terry" "age" "42"]

I'll say that's a respectable time... and the leading contestant 
:)

:-)

now if only i was 42 again...

But wait, there's more.... 

convert dataset to hash! and run ultimate-find again!

>> ultimate-find dataset "age"  2 3 100
ultimate find():  -> 0:00   2 matches found
== ["maximn" "age" "unknown" "terry" "age" "42"]

100 iterations not even registering

1000 iterations 0.40

OMG !

exactly

but I'm getting an odd deadlock here on some tests... hum...

I'm getting extremely slow results on dense tests...

interesting... im not too worried as density isn't a big issue with 
triple stores