Mailing List Archive: 49091 messages
  • Home
  • Script library
  • AltME Archive
  • Mailing list
  • Articles Index
  • Site search
 

[REBOL] Re: decimal woes

From: roland:hadinger:arcor at: 18-Jul-2003 18:51

Hi Maxim, decimal fractions like 0.02 are typically represented as binary fractions at the hardware level, in this case: 0.02 base 10 0.0_00001010001111010111_ recurring base 2 Due to the limited precision a roundoff error is introduced at the the 23rd, 53th or 80th bit of the mantissa (depending on the floating point format), which tends to accumulate when you do lots of additions and subtractions. One option would be to use fixed point arithmetic done with premultiplied numbers. Another option would be to rewrite the code: time: 0.0 time-start: 0.0 time-end: 1.0 repeat n 50 [ m: n / 50 time: time-start * (1 - m) + time-end * m probe time ] Depending on the number of iterations, this may (not necessarily!) be more accurate.