Mailing List Archive: 49091 messages
  • Home
  • Script library
  • AltME Archive
  • Mailing list
  • Articles Index
  • Site search

[REBOL] Benchmarking alternatives

From: joel:neely:fedex at: 6-Nov-2001 15:49

Hi, all, The comment was made a little while back (my apologies for not taking the time to track down the source) suggesting that EITHER would be probably be faster than IF/ELSE. Since I've been in a benchmarking mode lately, I went ahead and tackled that one... Based on a test of 100_000_000 iterations, the results seem fairly stable. I repeated the same suite of tests for two blocks (one empty and the other rather trivial), to try to factor out the time required for actual control function evaluation independent of the time for the controlled expression. The expression schemata in the left column were iterated via LOOP for the two blocks above the second and third columns, which give raw times. actual blocks ---------------- Schemata [] [v: 1] ----------------------- ------- ------- [?] 3.145 22.753 [do [?]] 66.615 84.832 [either true [?] [?]] 93.134 114.074 [if/else true [?] [?]] 101.686 124.398 [either false [?] [?]] 92.683 114.094 [if/else false [?] [?]] 101.857 124.339 Subtracting out the various bits of overhead, these numbers support the following conclusions about performance: 1) Neither decision function (EITHER or IF/ELSE) is biased in favor of a true or false condition. 2) EITHER takes approximately 35%-40% more time than DO for the same evaluated (controlled) block. 3) IF/ELSE takes approximately 10% more time than EITHER. While still in the benchmarking frame of mind, I pondered the fact that one can set a word to a block and then use that word wherever that block is required (e.g., LOOP numvar blkvar). One can also: * cause the block to be evaluated with DO (e.g., DO blkvar), * convert the block to a PAREN! value (which sort of automagically DOes itself), or * use the block as the body of a parameterless function with no local variables (as in DOES). Hmmm... four different ways to do the same thing? Let's see how they compare in performance! As before, there are several schemata, which can be tested with various block values to separate the cost of the schema from the cost of the controlled expression(s). The timings are as follows: actual blocks --------------------------- Schemata [] [v: 1] [v: v + 1] ---------- ------ ------- ---------- [?] 3.105 22.753 57.413 [do [?]] 69.069 85.713 121.504 (?) 15.062 35.090 76.951 func [][?] 31.385 49.181 90.390 Again subtracting out the various overheads, we find substantial amounts of variation. I probably need to run a much longer test, and try some other blocks to try to track down the variation. However, at a first glance, we find that: 1) On average, DOing a block takes about four and a half times as long as evaluating a paren constructed from the same block. 2) On average, evaluating a function (from DOES) takes about twice the amount of time as evaluating an equivalent paren. Conclusion? Instead of writing (for expression re-use) bv: [...some expressions...] ... ... do bv ... or dv: does [...some expressions...] ... ... dv ... we would prefer to write pv: to-paren [...some expressions...] ... ... pv ... if speed is of the essence. -jn- -- This sentence contradicts itself -- no actually it doesn't. -- Doug Hofstadter joel<dot>neely<at>fedex<dot>com