r3wp [groups: 83 posts: 189283]
  • Home
  • Script library
  • AltME Archive
  • Mailing list
  • Articles Index
  • Site search
 

World: r3wp

[Core] Discuss core issues

Steeve
18-Dec-2008
[11579]
i give you a script i used to profile the ideal size of the buffer 
used with read-io to have the best perfs.
On my computer the best size for the buffer is 8ko or 16ko.

REBOL []
f: open/seek/binary %large.dta

foreach len [64 128 256 1024 2048 4096 8192 10240 16384 32768 65536 
131072] [

	f/state/index: 0		;*** Problème quand on emploie read-io :
					;*** apparement c'est un bug, par défaut l'index est à 1
					;*** du coup, le premier octet n'est jamais lu
	

 buff: make binary! len + 1	;*** Encore un bizarerie, si le buffer 
 a exactement
					;*** la taille voulue, read-io lit un octet de moins
	n: 0
	recycle
	t: now/time/precise

 while [0 < read-io f buff len] [n: n + 1 clear buff f/state/index: 
 f/state/index + len]
	print [len tab v: now/time/precise - t tab v / n tab n]
]

close f
halt
[unknown: 5]
18-Dec-2008
[11580x3]
See your declaring the size of your buffer so wouldn't have seen 
the other problem.
you don't need the /binary switch with seek in 2.7.6
it is /binary by default.
Steeve
18-Dec-2008
[11583x2]
i know
in fact it's depending of the type of the buffer
[unknown: 5]
18-Dec-2008
[11585]
yeah that is true - which is very cool about REBOL.
Steeve
18-Dec-2008
[11586]
i noticed too, that your buffer must be size of the requested size 
+ 1 byte, if not you get some errors
[unknown: 5]
18-Dec-2008
[11587x3]
I copy data from a port using /seek and simply do-    to-block port-data 
and need no other conversion.
That is probably why it stopped at 15.
I think index was at 1
Steeve
18-Dec-2008
[11590]
the index is zero based, don't forget
[unknown: 5]
18-Dec-2008
[11591]
Nope - still reads just 15
BrianH
18-Dec-2008
[11592]
Requested size + 1 or more bytes.
[unknown: 5]
18-Dec-2008
[11593]
yeah I was checking empty buffer
Steeve
18-Dec-2008
[11594]
did you try the profiler script ?
[unknown: 5]
18-Dec-2008
[11595x4]
No, I'm in the middle of something else
I'll open another console  real quick
64   0:00    351909:41:29.709551615      0
128      0:00    351909:41:29.709551615      0
256      0:00:00.016     351909:41:29.709551615      0
1024     0:00    351909:41:29.709551615      0
2048     0:00    351909:41:29.709551615      0
4096     0:00    351909:41:29.709551615      0
8192     0:00    351909:41:29.709551615      0
10240    0:00    351909:41:29.709551615      0
16384    0:00    351909:41:29.709551615      0
32768    0:00    351909:41:29.709551615      0
65536    0:00    351909:41:29.709551615      0
131072   0:00    351909:41:29.709551615      0
pretty consistent
Steeve
18-Dec-2008
[11599]
something is wrong
BrianH
18-Dec-2008
[11600]
You need to print out the time to more precision.
Steeve
18-Dec-2008
[11601]
you need to use a big file as data
[unknown: 5]
18-Dec-2008
[11602x3]
Yeah I just pasted it
hold on
64   0:00:10.639     0:00:00.000020633   515625
128      0:00:05.444     0:00:00.000021116   257813
256      0:00:02.761     0:00:00.000021418   128907
1024     0:00:00.733     0:00:00.000022744   32227
2048     0:00:00.374     0:00:00.000023209   16114
4096     0:00:00.218     0:00:00.000027057   8057
8192     0:00:00.124     0:00:00.000030776   4029
10240    0:00:00.109     0:00:00.000033819   3223
16384    0:00:00.078     0:00:00.000038709   2015
32768    0:00:00.078     0:00:00.00007738    1008
65536    0:00:00.047     0:00:00.000093253   504
131072   0:00:00.063     0:00:00.00025   252
>>
BrianH
18-Dec-2008
[11605]
Looks like 64k is a good value for you.
Steeve
18-Dec-2008
[11606x2]
yeah it seems
but in fact, you should execute it several times, results can be 
different
BrianH
18-Dec-2008
[11608]
Particularly on Windows.
Steeve
18-Dec-2008
[11609x4]
in fact you can see that 16ko buffer would be a better choice (regarding 
to the first column)
(the third column)
you must do more tests
brb, i have to request some cigarettes and food
[unknown: 5]
18-Dec-2008
[11613]
Steeve, I'm finding that copy/part using 'at on /seek is working 
at more speed for me.
Steeve
18-Dec-2008
[11614]
you must have a special computer, it's not quiet logical and i have 
opposite results on my computer
[unknown: 5]
18-Dec-2008
[11615x2]
s: now/precise
a: open/direct/binary %blah
idx: 0
a/state/index: 0
buff: make binary! 16

loop 100000 [read-io a buff 16 a/state/index: a/state/index + 16 
clear buff]
close a
t: now/precise
difference t s
my datafile is %blah
Steeve
18-Dec-2008
[11617]
oh really ? ;-)
[unknown: 5]
18-Dec-2008
[11618x5]
== 0:00:01.934
That is what I get with read-io.
s: now/precise
a: open/seek %blah
idx: 0
loop 100000 [copy/part at a (idx: idx + 16) 16]
close a
t: now/precise
difference t s
what I get with copy/part:
== 0:00:01.747
Steeve
18-Dec-2008
[11623x2]
using a buffer of 16 bytes is YOUR problem, didn't you understand 
the result of the pofiling script
use a 16ko buffer instead (16ko = 16 * 1024)
[unknown: 5]
18-Dec-2008
[11625]
Yes, I did Steeve, but that is my point - I'm saying for MY purposes 
read-io i s not faster.
Oldes
18-Dec-2008
[11626]
Do you mean you MUST use only 16 bytes?
Steeve
18-Dec-2008
[11627]
but copy use a larger buffer that 16 bytes, you don't compare same 
things
[unknown: 5]
18-Dec-2008
[11628]
I'm comparing the same for what I need.