Mailing List Archive: 49091 messages
  • Home
  • Script library
  • AltME Archive
  • Mailing list
  • Articles Index
  • Site search
 

[REBOL] Re: mathematical rounding...

From: media:quazart at: 6-Nov-2001 14:02

HI, Following the rounding question I have done some tests... and Tought I'd send this little note to everyone... Although this seems to be normal IEEE sanctioned behaviour, I have noticed that producing a floating point calculation with more than 15 characters of precisions as in: 1.499999999999999 will affect the actuall value of the flating-point number. evaluating the above actually returns 1.5 ... try it, you'll see ! so the above rounded return 2 not 1 !!! adding 0.5 to the above will cause an overflow to occur and 1.999999999999999... actually becomes 2.0 ! thanks for that excellent reference go to Robert Hamilton... This behaviour is mentioned and somewhat explained... in some of the pages when you dig a little bit... it seems to have to do with binary translation of decimal values (or something like that, excuse my gross interpretation)... Just thought I'd point it out since it isn't such an obvious side-effect to debug when coding... And Like many if you I'm not a scientific maths expert... making it THAT much less obvious... Ciao! -MAx