Mailing List Archive: 49091 messages
  • Home
  • Script library
  • AltME Archive
  • Mailing list
  • Articles Index
  • Site search

[REBOL] Re: RFC: Cross-language benchmark proposal

From: joel:neely:fedex at: 9-Nov-2002 8:09

Hi, Carl, and all, I think we're making this WAAAAAY harder than it needs to be! (And I'll accept my full share of responsbility... ;-) Carl Read wrote:
> On 09-Nov-02, Ed Dana wrote: > > > The algorithm for any program should be specified in simple > > English. It should be void of any technical references nor > > should it suggest any technical solutions. >
The issue which concerns me (and which I accept responsibility for not communicating clearly) is that we're trying to compare *language*implementations* and not *programmers*. Therefore we have to be clear about how much latitude is allowed to the author of a test solution, and only compare solutions that are fairly comparable. Suppose we had a task specified as follows: - Generate a list/array of 10000 random integers, each between 1 and 1000000; - Sort the array into ascending order; - Sort the result into descending order. Now suppose we get the following submissions: Program Programmer Algorithm Language ------- ---------- ------------- -------- A Alice bubblesort REBOL B Bob quicksort REBOL C Clara built-in SORT REBOL D Don bubblesort Perl E Ethyl build-in SORT Perl We could compare A and D to get an idea of how the languages compare in their handling of looping and array/block access speeds. We could compare C and E to get an idea of how the languages compare in their fastest possible sorting mechanisms (I'm assuming that the built-in SORT facilities are faster than someone could rewrite in interpreted code). But comparing B with *any* other submission doesn't IMHO give us any meaningful compare-REBOL-with-Perl information (although it *does* tell us that Bob wrote a more sophisticated algorithm than Alice, and it might tell us just how much gain a REBOL programmer could realize by using SORT instead of a home-grown function). Given those five submissions, I'd suggest that we probably ought to split that hypothetical problem above into two tasks: 1) Solve the sorting problem above, using bubblesort according to the following pseudo-code specification ...; and 2) Solve the sorting problem above, using the fastest sorting means you can come up with. Then I'd view A and D as solutions to (1); C and E would make good solutions to (2); I don't think it would be reasonable to compare e.g. A with E (or B with D) and think I was thereby making a fair compare/contrast examination of REBOL and Perl.
> > For example, the algorithm for the RandomCopy procedure I > > mentioned earlier is: > > * Specify Source Directory > > * Specify Target Directory or File > > > 1 Get all the files in the Source Directory. > > 2 Choose one file randomly. > > 3 Copy it to the Target File or Directory. > > > Stating it this way keeps it at a "logical" level and should > > avoid any "subjective" issues as the only constraints here > > are the resources being utilized. I.e. Files and directories. > > Okay. And I'd write that something like this, (though with a filled > header of course), ... > > rebol [] > dir: %test-dir/ > file: %test-dir/random-file > write/binary file read/binary join dir random/only read dir >
That seems quite reasonable for the problem as stated. Now, let's suppose that some novice C programmer wrote a submission that was structured as follows: 1) Recursively traverse the entire file system from the root directory, performing the following for each fully-pathed file name encountered: 1.1) If the encountered name is a complete prefix match to the specified source directory, and there are no more slashes in the remainder of the encountered name, allocate a buffer to hold the name, copy the name into the buffer, and store a pointer to the buffer into the array of file names. Oh, by the way, if that filename array is full, allocate a new one twice the size, copy all existing pointers into the new one, and then append the new name pointer. 2) After traversing the entire file system, shuffle the array of file name pointers, using a bubble sort algorithm with a comparison function that returns random results. 3) Open the file whose name is in the middle of the shuffled array. ... I've already gone on too long, but pretend that the rest of the program is as BBD as the above. Now, what would we learn about C and REBOL by comparing that program with your three-liner? (For extra credit, what would we learn about the kind of twisted mind that could come up with such an obviously wrong-headed algorighm? ;-) -jn- -- ; Joel Neely joeldotneelyatfedexdotcom REBOL [] do [ do func [s] [ foreach [a b] s [prin b] ] sort/skip do function [s] [t] [ t: "" foreach [a b] s [repend t [b a]] t ] { | e s m!zauafBpcvekexEohthjJakwLrngohOqrlryRnsctdtiub} 2 ]