[REBOL] Re: Profiling Rebol API to DyBASE
From: knizhnik:garret:ru at: 19-Dec-2003 1:45
I improve my own "record".
Looks like the best solution (not only for this example) is:
n: 200000
h: make hash! 101
cache-size: 101
cache-used: 0
start: now/time/precise
repeat i n [
oid: random n
obj: select h oid
if none? obj [
obj: make object! [__oid__: oid]
if cache-used = cache-size [
cache-size: cache-size * 2
new-hash: make hash! cache-size
insert tail new-hash h
h: new-hash
]
insert insert tail h oid obj
cache-used: cache-used + 1
]
if (i // 100) = 0 [h: make hash! 101 cache-size: 101 cache-used: 0]
]
print ["Elapsed time for adding" n "records" (now/time/precise - start)]
Friday, December 19, 2003, 1:22:13 AM, you wrote:
KK> Hello Gregg,
KK> I was able to isolate the problem.
KK> The following script shows almost the same time as testindex.r
KK> searching for 200000 objects.
KK> n: 200000
KK> h: make hash! n
KK> start: now/time/precise
KK> repeat i n [
KK> oid: random n
KK> obj: select h oid
KK> if none? obj [
KK> obj: make object! [__oid__: oid]
KK> insert insert tail h oid obj
KK> ]
KK> if (i // 100) = 0 [clear h]
KK> ]
KK> print ["Elapsed time for adding" n "records" (now/time/precise - start)]
KK> At my computer execution of this script takes about 70 seconds.
KK> By replacing it with:
KK> n: 200000
KK> h: make hash! n
KK> l: make block! n
KK> start: now/time/precise
KK> repeat i n [
KK> oid: random n
KK> pos: find h oid
KK> either none? pos [
KK> obj: make object! [__oid__: oid]
KK> insert tail h oid
KK> insert tail l obj
KK> ] [obj: pick l index? pos]
KK> if (i // 100) = 0 [clear h clear l]
KK> ]
KK> print ["Elapsed time for adding" n "records" (now/time/precise - start)]
KK> I was able to reduce execution time till 33 seconds.
KK> Are there some better ideas how to improve performance of this peace
KK> of code?
KK> Thursday, December 18, 2003, 11:46:41 PM, you wrote:
GI>> Hi Konstantin,
KK>>> So, during index search 1.5 minutes from 2 were spent in lookup
KK>>> function. And 14 seconds takes searching index itself.
KK>>> From these 1.5 minutes most of the time was spent in this line:
KK>>> obj: select obj-by-oid-map oid
GI>> A quick test seems to show that the SELECT part of the lookup is
GI>> faster for smaller numbers of records (e.g. 10,000), and gets
GI>> progressively slower as the numbers increase. That is, it starts out
GI>> faster than the FETCH and MAKE parts and ends up a lot slower than
GI>> they are with larger numbers of records.
GI>> -- Gregg
KK> --
KK> Best regards,
KK> Konstantin mailto:[knizhnik--garret--ru]
--
Best regards,
Konstantin mailto:[knizhnik--garret--ru]
