Mailing List Archive: 49091 messages
  • Home
  • Script library
  • AltME Archive
  • Mailing list
  • Articles Index
  • Site search

[REBOL] Re: decimal woes

From: maximo:meteorstudios at: 18-Jul-2003 13:28

reply in-line PS: check out the e-mail replacement... it sometimes work when sites ask for passwords... the sysadmin receives their own spam ;-)
> -----Original Message----- > From: Roland Hadinger [[root--localhost--com]] > Hi Maxim, > > decimal fractions like 0.02 are typically represented as binary > fractions at the hardware level, in this case: > > 0.02 base 10 > 0.0_00001010001111010111_ recurring base 2 > > Due to the limited precision a roundoff error is introduced at the > the 23rd, 53th or 80th bit of the mantissa (depending on the floating > point format), which tends to accumulate when you do lots of > additions and subtractions.
hum, wish I had a major in maths ;-) I understand that fp usually accumulates errors (in cgi its a big problem with 8-bit/channel gfx). But, if the value is going to be off by a small amount, shouldn't it be visible with a print. I mean, If, in the loop, the value to add to is printed as 0.92 I expect it to behave similarly as if I do 0.92 + 0.02 in the console... no? Do you think this means that print and probe do not contain all digits of the internal hw decimal value? so that when it prints 0.92 it only looked at the 15 first digits and there was still some data past that, which only spills off after repeated (and expected) rounding errors? if so, does anyone know how to build a print-xtended-decimal function?
> One option would be to use fixed point arithmetic done with > premultiplied numbers.
That's exaclty what I did, and it works (but its a bit ugly when you have built-in fp). Working in the 'assumed' resolution of thousanths of a secound and dividing by 1000 when doing comparisons with the supplied data, does work.
> Another option would be to rewrite the code: > > time: 0.0 > time-start: 0.0 > time-end: 1.0 > > repeat n 50 [ ...]
alas, in the real code I do not know the lenght of the series (searching through data) and must exit at tail? of series, so I cant implement it that way.