[REBOL] Benchmarking alternatives
From: joel:neely:fedex at: 6-Nov-2001 15:49
Hi, all,
The comment was made a little while back (my apologies for not
taking the time to track down the source) suggesting that EITHER
would be probably be faster than IF/ELSE. Since I've been in a
benchmarking mode lately, I went ahead and tackled that one...
Based on a test of 100_000_000 iterations, the results seem fairly
stable. I repeated the same suite of tests for two blocks (one
empty and the other rather trivial), to try to factor out the
time required for actual control function evaluation independent
of the time for the controlled expression. The expression schemata
in the left column were iterated via LOOP for the two blocks above
the second and third columns, which give raw times.
actual blocks
----------------
Schemata [] [v: 1]
----------------------- ------- -------
[?] 3.145 22.753
[do [?]] 66.615 84.832
[either true [?] [?]] 93.134 114.074
[if/else true [?] [?]] 101.686 124.398
[either false [?] [?]] 92.683 114.094
[if/else false [?] [?]] 101.857 124.339
Subtracting out the various bits of overhead, these numbers support
the following conclusions about performance:
1) Neither decision function (EITHER or IF/ELSE) is biased in favor
of a true or false condition.
2) EITHER takes approximately 35%-40% more time than DO for the same
evaluated (controlled) block.
3) IF/ELSE takes approximately 10% more time than EITHER.
While still in the benchmarking frame of mind, I pondered the fact
that one can set a word to a block and then use that word wherever
that block is required (e.g., LOOP numvar blkvar). One can also:
* cause the block to be evaluated with DO (e.g., DO blkvar),
* convert the block to a PAREN! value (which sort of automagically
DOes itself), or
* use the block as the body of a parameterless function with no
local variables (as in DOES).
Hmmm... four different ways to do the same thing? Let's see how
they compare in performance!
As before, there are several schemata, which can be tested with
various block values to separate the cost of the schema from the
cost of the controlled expression(s). The timings are as follows:
actual blocks
---------------------------
Schemata [] [v: 1] [v: v + 1]
---------- ------ ------- ----------
[?] 3.105 22.753 57.413
[do [?]] 69.069 85.713 121.504
(?) 15.062 35.090 76.951
func [][?] 31.385 49.181 90.390
Again subtracting out the various overheads, we find substantial amounts
of variation. I probably need to run a much longer test, and try some
other blocks to try to track down the variation. However, at a first
glance, we find that:
1) On average, DOing a block takes about four and a half times as long
as evaluating a paren constructed from the same block.
2) On average, evaluating a function (from DOES) takes about twice the
amount of time as evaluating an equivalent paren.
Conclusion? Instead of writing (for expression re-use)
bv: [...some expressions...]
...
... do bv ...
or
dv: does [...some expressions...]
...
... dv ...
we would prefer to write
pv: to-paren [...some expressions...]
...
... pv ...
if speed is of the essence.
-jn-
--
This sentence contradicts itself -- no actually it doesn't.
-- Doug Hofstadter
joel<dot>neely<at>fedex<dot>com