[REBOL] Re: strange results with decimal!
From: philb:upnaway at: 10-Jan-2002 22:43
I would think that the problem is the way computers hold decimals.
As a decimal is held as a binary representaion of the decimal in a finite number of digits
(usually 16,32 or 64) in most cases it cannot hold a decimal exactly and some rounding
We dont know how internally Rebol hold decimal numbers but as its C based it may well
hold them in base 2 decimal form.
9.3 = 9 * 10 + 3 * 0.1
has the binary form
1*2^3 + 0*2^2 + 0*2^1 + 1*2^0 + 0*2^-1 + 1*2^-2 + 0*2^-3 + 0*2^-4 + 1*2^-5
As this is a recurring binary decimal some rounding will occur so the decimal number
is not held exactly.
Consider the following (typed at the console)
>> 0.33333333333333 + 0.333333333333333 + 0.333333333333333
This indicates that the number 0.33333333333333 was truncated.
I would suggest that one or both of numbers in your example was truncated and gave you
the unexpected result.
=== Original Message ===
thanks for your reply. If devision or multiplication is involvolved i could
follow your samples. Perhaps i should be more detailed, as i am only using
addition and substraction.
15888;some text;1.000,50;1.000,50;some text
15889;some text;50,49;50,48;some text
As the first line contains german values, i have to parse out the "." and
the convert the field to a decimal like 1.000,50 to 1000,50.
After that i add the values to a sum with a foreach loop. the result is:
Now i want the difference to be displayed.
print sum-a - sum-b
The result is, even with this example, "9.99999999999091E-3" instead of
If you just type in the lines with the sums and the print statement you can
follow the example.
Is that the same problem as you described?