[REBOL] Re: strange results with decimal!
From: joel:neely:fedex at: 10-Jan-2002 10:10
Hi, Thorsten,
Thorsten Moeller wrote:
> sum-a: 1050,99
> sum-b: 1050,98
>
> Now i want the difference to be displayed.
>
> print sum-a - sum-b
>
> The result is, even with this example, "9.99999999999091E-3"
> instead of 0,01.
>
> If you just type in the lines with the sums and the print
> statement you can follow the example.
>
Or even
>> 1050.99 - 1050.98
== 9.99999999999091E-3
> Is that the same problem as you described?
>
Yes, because converting the string "1050.99" to a number actually
does require division; it's equal to 105099 / 100 !
>> (105099 / 100) - (105098 / 100)
== 9.99999999999091E-3
Just as 1/3 in decimal is 0.333333333333333333333333333333333333...
remember that 1/5 in binary is 0.00110011001100110011001100110011...
and therefore 1/10 is 0.00011001100110011001100110011001100110011...
The only two-digit decimal fractions that have exact representations
as binary fractions are 0.25, 0.50, and 0.75; any other number of
hundredths
will encounter round-off, even if you're only taking
a string representation of it into decimal.
-jn-
