r3wp [groups: 83 posts: 189283]
  • Home
  • Script library
  • AltME Archive
  • Mailing list
  • Articles Index
  • Site search
 

World: r3wp

[Profiling] Rebol code optimisation and algorithm comparisons.

Maxim
19-May-2010
[197]
you can negate collisions by building two checksums out of different 
properties of you data and merging them.
Terry
19-May-2010
[198x2]
fair enough.. not running bank accounts with this thing
the other issue is the time it takes to build the checksum vs brute 
force
Steeve
19-May-2010
[200x2]
but it will be 100 or 1000 times faster, then to access the data 
using an index.
your actual trial to make a lookup with foreach or find+loop is insanly 
slow by comparison
Sunanda
19-May-2010
[202]
Got to decide what is more important:
-- time to build data structure
-- time to update it (add/remove on the fly)
-- time to search it

And build data structures optimized to your priorities. There is 
no one true solution, just the best match for the situation at hand.
Steeve
19-May-2010
[203]
*current trial
Terry
19-May-2010
[204]
ok.. here's an example.. 

take this simple rdf triple   "Tweety" "isa" "Canary"
How would create 3 indexes to manage it and 10,000,000 like it?
Steeve
19-May-2010
[205x2]
It's the problem, I think Terry can't decide :-)
Ok, I give it to you...
Terry
19-May-2010
[207x2]
Tweety
 "age" "75"
Steeve
 "isa" "Rebol"
Steeve
 "age" "unknown"
I have a system working now that's fast enough.. but I'm a speed 
junkie.. there must be a BEST way (not better... BEST)
Steeve
19-May-2010
[209x4]
First i add the triple in the triples store (a simple block)
The, I recovers its index from the block (actually it's the last 
one)
And i use this index as the value for the 3 indexes, and i create 
the keys+value
Tweety
: index
Isa
: index
Canary
: index
that's all...
Terry
19-May-2010
[213]
yeah, but ...
Steeve
19-May-2010
[214]
Perhaps the explanations are not clear, but it's pretty clear in 
my head ;-)
Terry
19-May-2010
[215x2]
>> ie: ["Tweety" "isa" "canary"]
== ["Tweety" "isa" "canary"]
>> index? find ie "Tweety"
== 1

Great.. now what
( only in Rebol does ie: actually mean ie: )
Steeve
19-May-2010
[217x2]
(added In index1) "Tweety"= index
(added In index2) "Isa"= index
(added In index3) "Canary"= index
(added In index1) "Tweety"= index
(added In index2) "Isa"= index
(added In index3) "Canary"= index
Terry
19-May-2010
[219]
so now i want to query, say , everything that is "age" "75" (like 
above)
Steeve
19-May-2010
[220x4]
by doing, index2/"age" you'll get all the triple having the verb 
"age"
with index3/"75", you'll get those with 75 as the target
then intersect the 2 blocks
I just think you don't get it, but I know my explanation sucks, sorry.
Terry
19-May-2010
[224]
i get it.
Sunanda
19-May-2010
[225]
Ron Everett presented  a database that did much of what you want 
at DevCon2007. The live discussion is here:

   http://www.rebol.org/aga-display-posts.r?offset=0&post=r3wp500x1919
The video of the presentation may be on qtask.
Pekr
19-May-2010
[226x2]
yeah, associative stuff :-)
Senteces from Lazysoft was another product of such kind ...
Terry
19-May-2010
[228]
i remember that
Steeve
19-May-2010
[229x2]
http://www.rebol.net/cgi-bin/r3blog.r?view=0161
And my comment...

it's remember me the IDE Plex(obsydian) in the nineties. it used 
widely the concept of triples (tuples) to modelize applications and 
databases.
Terry
19-May-2010
[231x3]
hmm, i thought i got it. :( now I'm lost in block hell
nope.. i got it again :)
Should have listened to my mother and became a lawyer.
Pekr
19-May-2010
[234]
why :-)
Terry
19-May-2010
[235x3]
Shouldn't Intersect take an block of blocks?
ie: intersect [ blk1 blk2 blk3.. ]
Works great Steeve. I haven't benchmarked it yet, but given the whole 
thing can sit in hashes or maps! , i would say the performance should 
be spectacular
Gregg
19-May-2010
[238]
Terry, I think INTERSECT is fine the way it is, and it's easy to 
wrap if you want.

    fold: func [
        series [series!]
        fn     [any-function!]
        /local value
    ][
        value: pick series 1
        foreach item next series [value: fn value item]
    ]

    intersect-all: func [
        series [block!] "A block of series values to intersect"
    ][
        fold series :intersect
    ]
Terry
19-May-2010
[239]
I knew someone would comback with a function.. might as well have 
been you Gregg.
I like PHP's array_merge()
Maxim
19-May-2010
[240]
steeve, REBOL doesn't support path with strings, and furthermore, 
it would only return the first index, if you used it within a paren.


so I'd really like you to give a small snippet of code with your 
idea, I am curious about your method... cause I don't see *how* what 
you say works.
Steeve
20-May-2010
[241]
I would use map! (or hash!) as indexes and a key would contain one 
or several triples (inside a block)
verb/"age": [ index indexn ...]
Ladislav
20-May-2010
[242]
Max, Steeve proposed the datastructure so, that the above feach operations 
become elementary ("instantaneous").
Steeve
20-May-2010
[243]
Yeah !!! :)
Terry
20-May-2010
[244x2]
Here's what I did..
ie: [["Steeve" "isa" "person"]["Steeve" "age" "42"]["Tweety" "isa" 
"Canary"]["Tweety" "age" "75"]["Chirp" "isa" "Canary"]]
i1: ["Steeve" [1 2] "Tweety" [3 4] "Chirp" [5]]
i2: ["isa" [1 3 5] "age" [2 4]]
i3: ["person" [1] "42" [2] "Canary" [3 5]]


rocket: func [it][
    st: now/precise

	while[it > 0][
		s: i1/("Steeve")
		p: i2/("isa")

		res1: intersect s p
		it: it - 1
	]
	prin join " -> " difference now/precise st
]
Steeve
20-May-2010
[246]
Yeah !!! You got it :)