[REBOL] Re: RFC: Cross-language benchmark proposal
From: greggirwin:mindspring at: 7-Nov-2002 12:38
Joel et al,
<< Let's start small, and add "features" as resources become available
and reader interest demands. >>
Yes!
<<
Tasks: A subset of the benchmarks on the "Shootout" page (the
ones that can be run in REBOL).
>>
Yes!
<<
Languages: C, Java, Perl, Python, REBOL
>>
What about VB? *Lots* of people use it, though it's not cross-platform.
<<
Coding: Starting with the published versions on "Shootout", with
open contribution from the REBOL community for REBOL
versions in two flavors:
Standard: Submissions must adhere to the "same thing"
rule from "Shootout".
Custom: Any approach that gets the correct results.
>>
As contributions are made, would custom entries for other languages be
allowed as well? Maybe a better question is: is the goal to show off REBOL,
even at the expense of other language proponents crying "foul" because it
gets special treatment?
These are really two different types of benchmarks, right? The Standard
version is an algorithm benchmark; how a language performs using a specified
algorithm. The Custom version is a "result oriented" benchmark. Just
thinking about how to organize things to make that clear.
<<
Solutions: Must take the form of an object with two methods:
Rationale: This will allow use of a single test harness
(see below) to gather consistent stats.
>>
I haven't looked at the tests to know if N is sufficient for all purposes.
Do you envision anything more elaborate being required?
<<
Partipants: Anybody who wants to contribute cycles, under the following
constraints:
Tasks: All results for a given task (run in multiple
languages) must be submitted at one time.
>>
So if I want to contribute, does this mean I need to write the solution in
each language, or is this just for test *result* submissions? What
requirements will there be for implementation contributions? E.g. the
Ackermann example has shown that there may be multiple approaches taken in
REBOL.
--Gregg
