[REBOL] Re: mathematical rounding...
From: media:quazart at: 6-Nov-2001 14:02
Following the rounding question I have done some tests... and Tought I'd
send this little note to everyone... Although this seems to be normal IEEE
I have noticed that producing a floating point calculation with more than 15
characters of precisions as in:
will affect the actuall value of the flating-point number.
evaluating the above actually returns 1.5 ... try it, you'll see !
so the above rounded return 2 not 1 !!!
adding 0.5 to the above will cause an overflow to occur and
1.999999999999999... actually becomes 2.0 !
thanks for that excellent reference go to Robert Hamilton... This behaviour
is mentioned and somewhat explained... in some of the pages when you dig a
little bit... it seems to have to do with binary translation of decimal
values (or something like that, excuse my gross interpretation)...
Just thought I'd point it out since it isn't such an obvious side-effect to
debug when coding... And Like many if you I'm not a scientific maths
expert... making it THAT much less obvious...