[REBOL] Re: strange results with decimal!
From: philb:upnaway at: 10-Jan-2002 22:43
Hi Thorsten,
I would think that the problem is the way computers hold decimals.
As a decimal is held as a binary representaion of the decimal in a finite number of digits
(usually 16,32 or 64) in most cases it cannot hold a decimal exactly and some rounding
occurs.
We dont know how internally Rebol hold decimal numbers but as its C based it may well
hold them in base 2 decimal form.
For example
9.3 = 9 * 10 + 3 * 0.1
has the binary form
1001.01001 (recurring)
meaning
1*2^3 + 0*2^2 + 0*2^1 + 1*2^0 + 0*2^-1 + 1*2^-2 + 0*2^-3 + 0*2^-4 + 1*2^-5
As this is a recurring binary decimal some rounding will occur so the decimal number
is not held exactly.
Consider the following (typed at the console)
>> 0.33333333333333 + 0.333333333333333 + 0.333333333333333
== 0.999999999999996
This indicates that the number 0.33333333333333 was truncated.
I would suggest that one or both of numbers in your example was truncated and gave you
the unexpected result.
Cheers Phil
=== Original Message ===
Hi Joel,
thanks for your reply. If devision or multiplication is involvolved i could
follow your samples. Perhaps i should be more detailed, as i am only using
addition and substraction.
File sheme:
15888;some text;1.000,50;1.000,50;some text
15889;some text;50,49;50,48;some text
As the first line contains german values, i have to parse out the "." and
the convert the field to a decimal like 1.000,50 to 1000,50.
After that i add the values to a sum with a foreach loop. the result is:
sum-a: 1050,99
sum-b: 1050,98
Now i want the difference to be displayed.
print sum-a - sum-b
The result is, even with this example, "9.99999999999091E-3" instead of
0,01.
If you just type in the lines with the sums and the print statement you can
follow the example.
Is that the same problem as you described?
Thorsten
