Mailing List Archive: 49091 messages

## [REBOL] Re: strange results with decimal!

### From: philb:upnaway at: 10-Jan-2002 22:43

```
Hi Thorsten,

I would think that the problem is the way computers hold decimals.

As a decimal is held as a binary representaion of the decimal in a finite number of digits
(usually 16,32 or 64) in most cases it cannot hold a decimal exactly and some rounding
occurs.

We dont know how internally Rebol hold decimal numbers but as its C based it may well
hold them in base 2 decimal form.
For example
9.3 = 9 * 10 + 3 * 0.1
has the binary form
1001.01001 (recurring)
meaning
1*2^3 + 0*2^2 + 0*2^1  + 1*2^0 + 0*2^-1 + 1*2^-2 + 0*2^-3 + 0*2^-4 + 1*2^-5

As this is a recurring binary decimal some rounding will occur so the decimal number
is not held exactly.

Consider the following (typed at the console)

>> 0.33333333333333 + 0.333333333333333 + 0.333333333333333
== 0.999999999999996

This indicates that the number 0.33333333333333 was truncated.

I would suggest that one or both of numbers in your example was truncated and gave you
the unexpected result.

Cheers Phil

=== Original Message ===

Hi Joel,

thanks for your reply. If devision or multiplication is involvolved i could
follow your samples. Perhaps i should be more detailed, as i am only using

File sheme:

15888;some text;1.000,50;1.000,50;some text
15889;some text;50,49;50,48;some text

As the first line contains german values, i have to parse out the "." and
the convert the field to a decimal like 1.000,50 to 1000,50.

After that i add the values to a sum with a foreach loop. the result is:

sum-a: 1050,99
sum-b: 1050,98

Now i want the difference to be displayed.

print sum-a - sum-b

The result is, even with this example, "9.99999999999091E-3" instead of
0,01.

If you just type in the lines with the sums and the print statement you can