# strange results with decimal!

### [1/17] from: moeller_thorsten::gmx::de at: 10-Jan-2002 11:47

Hi rebolers,
i have a script that gives to produce an output when there is a difference
between two values. Therefore i read a textfile, convert some fields to
decimal values and add them together to one or another sum. if a specific condition
comes true they are are compared should be 0 (checked manually with a small
amount). For some reason i get results like "-1.3476653466456E-12".
Does anybody understand this?
Any suggestions will be appreciated!!
Thorsten
--
--------------------------------------------------------------------------------------
Few can foresee whither their road will lead them,
till they come to it's
end.

### [2/17] from: joel:neely:fedex at: 9-Jan-2002 23:59

Hi, Thorsten,
Thorsten Moeller wrote:

> Hi rebolers,
> i have a script that gives to produce an output when there is a

<<quoted lines omitted: 5>>

> Does anybody understand this?
> Any suggestions will be appreciated!!
I strongly suspect you're falling victim to binary round-off.
In general it is A Bad Idea to perform floating-point (decimal!)
calculations and then expect to get an exact equality; rather
you should check for the difference between two values being
very small. For example, instead of
a: ...lots of decimal! calculation...
b: ...more decimal! calculation or literal...
if a = b ...
you should try
a: ...as above...
b: ...as above...
if 1.0e-10 > a - b ...
This is a well-known phenomenon in numerical analysis; to see
why it's the case, imagine performing the following decimal
calculation on a calculator with a fixed number of digits (or
with pencil and paper using only e.g., 5 digits):
a: 1.0 / 3.0
b: 1.0 / 3.0
c: 1.0 / 3.0
print a + b + c
If you use exactly 5 digits of precision, you'll get
a = 0.3333
b = 0.3333
c = 0.3333
a + b + c = 0.9999
since 1/3 does not have a terminating fractional expansion.
The only divisors which produce terminating fractions are
those containing only powers of 2 and/or 5, since those are
the only divisors of 10.
Since nothing but 2 divides 2, the only terminating binary
fractions are those with divisors of powers of two. Since
we use decimal externally, the conversion to binary and
subsequent calculation are likely to produce the phenomenon.
The reason why most desk calculators will perform the above
computation and return 1.0 is that they keep additional
hidden
digits beyond what is displayed, and then round to
the display size. If we had kept 6 digits and rounded to 5
digits for display, we'd have gotten a result of
0.99999
and rounded that to the (displayable)
1.0000
HTH!
-jn-
--
Shouting at the ground doesn't enable it to hear any better.
-- Henry Warwick
joel^dot^FIX^PUNCTUATION^neely^at^fedex^dot^com

### [3/17] from: moeller_thorsten:gmx at: 10-Jan-2002 13:39

Hi Joel,
thanks for your reply. If devision or multiplication is involvolved i could
follow your samples. Perhaps i should be more detailed, as i am only using
addition and substraction.
File sheme:
15888;some text;1.000,50;1.000,50;some text
15889;some text;50,49;50,48;some text
As the first line contains german values, i have to parse out the "." and
the convert the field to a decimal like 1.000,50 to 1000,50.
After that i add the values to a sum with a foreach loop. the result is:
sum-a: 1050,99
sum-b: 1050,98
Now i want the difference to be displayed.
print sum-a - sum-b
The result is, even with this example, "9.99999999999091E-3" instead of
0,01.
If you just type in the lines with the sums and the print statement you can
follow the example.
Is that the same problem as you described?
Thorsten

### [4/17] from: rebol665:ifrance at: 10-Jan-2002 14:37

Hi,
Decimal are store in IEEE format which is basically made of a binary
mantissa 0,xxxxxxxx and an exponent.
Converting decimal so binary leads to minor error.
To understand this try to convert 0,75 and 0,80 in binary.
0,75 is 0,11 in base two ( 0,1 + 0,01 or 0,5 + 0,25)
To understand this, you must remember that the comma moves to the right to
multiply by two ( 1 is 0,1 times two ). So 0,1 binary is 0,5 decimal. You
can use this to verify that 0,75 decimal is 0,11 binary. 0,11 binary with
two right moves gives 11 binary which is 3 decimal ( O,75 * 2 * 2 ).
If you try to do this with 0,80 you will find it never ends round because
you have to builds 0,8 decimal by addition of number like 0,5 0,25 0,125
0,0625 0,03125 etc.
So 0,8 in binary is like 1/3 in decimal
Patrick

### [5/17] from: philb:upnaway at: 10-Jan-2002 22:43

Hi Thorsten,
I would think that the problem is the way computers hold decimals.
As a decimal is held as a binary representaion of the decimal in a finite number of digits
(usually 16,32 or 64) in most cases it cannot hold a decimal exactly and some rounding
occurs.
We dont know how internally Rebol hold decimal numbers but as its C based it may well
hold them in base 2 decimal form.
For example
9.3 = 9 * 10 + 3 * 0.1
has the binary form
1001.01001 (recurring)
meaning
1*2^3 + 0*2^2 + 0*2^1 + 1*2^0 + 0*2^-1 + 1*2^-2 + 0*2^-3 + 0*2^-4 + 1*2^-5
As this is a recurring binary decimal some rounding will occur so the decimal number
is not held exactly.
Consider the following (typed at the console)

>> 0.33333333333333 + 0.333333333333333 + 0.333333333333333

== 0.999999999999996
This indicates that the number 0.33333333333333 was truncated.
I would suggest that one or both of numbers in your example was truncated and gave you
the unexpected result.
Cheers Phil
=== Original Message ===
Hi Joel,
thanks for your reply. If devision or multiplication is involvolved i could
follow your samples. Perhaps i should be more detailed, as i am only using
addition and substraction.
File sheme:
15888;some text;1.000,50;1.000,50;some text
15889;some text;50,49;50,48;some text
As the first line contains german values, i have to parse out the "." and
the convert the field to a decimal like 1.000,50 to 1000,50.
After that i add the values to a sum with a foreach loop. the result is:
sum-a: 1050,99
sum-b: 1050,98
Now i want the difference to be displayed.
print sum-a - sum-b
The result is, even with this example, "9.99999999999091E-3" instead of
0,01.
If you just type in the lines with the sums and the print statement you can
follow the example.
Is that the same problem as you described?
Thorsten

### [6/17] from: joel:neely:fedex at: 10-Jan-2002 10:10

Hi, Thorsten,
Thorsten Moeller wrote:

> sum-a: 1050,99
> sum-b: 1050,98

<<quoted lines omitted: 4>>

> If you just type in the lines with the sums and the print
> statement you can follow the example.
Or even

>> 1050.99 - 1050.98

== 9.99999999999091E-3

> Is that the same problem as you described?
>

Yes, because converting the string "1050.99" to a number actually
does require division; it's equal to 105099 / 100 !

>> (105099 / 100) - (105098 / 100)

== 9.99999999999091E-3
Just as 1/3 in decimal is 0.333333333333333333333333333333333333...
remember that 1/5 in binary is 0.00110011001100110011001100110011...
and therefore 1/10 is 0.00011001100110011001100110011001100110011...
The only two-digit decimal fractions that have exact representations
as binary fractions are 0.25, 0.50, and 0.75; any other number of
hundredths
will encounter round-off, even if you're only taking
a string representation of it into decimal.
-jn-

### [7/17] from: holger:rebol at: 10-Jan-2002 8:59

On Thu, Jan 10, 2002 at 10:43:14PM +0800,

**[philb--upnaway--com]**wrote:> Hi Thorsten,
>
> I would think that the problem is the way computers hold decimals.
> As a decimal is held as a binary representaion of the decimal in a finite number of
digits (usually 16,32 or 64) in most cases it cannot hold a decimal exactly and some
rounding occurs.
>
> We dont know how internally Rebol hold decimal numbers but as its C based it may well
hold them in base 2 decimal form.

REBOL uses whatever format the platform it is running on uses.
Typically this is IEEE double precision, i.e. base-2.
--
Holger Kruse

**[holger--rebol--com]**### [8/17] from: greggirwin:mindspring at: 10-Jan-2002 13:43

Hi Thorsten,
As Joel pointed out, you can compare decimal numbers using a small delta to
account for rounding errors. Another approach is to manually round the
numbers and compare the rounded results. Below is a ROUND function (and a
supporting MOD function), based on Ladislav's work, if you want to go that
route.
--Gregg
; Ladislav Mecir, Gregg Irwin (minor adjustment)
mod: func [
{Compute a remainder.}
value1 [number! money! time!] {The dividend}
value2 [number! money! time!] {The divisor}
/euclid {Compute a non-negative remainder such that: a = qb + r and
r < b}
/local r
] [
either euclid [
either negative? r: value1 // value2 [r + abs value2] [r]
;-- Alternate implementation
;value1 // value2 + (value2: abs value2) // value2
][
value1 // value2
]
]
;-- Note: to-interval does mod-like rounding. If the interval you
; specify is not evenly divisble into your base, the result
; may not be what you expect. E.g. round/to-interval 133 30
; will round to 120, not 130, because 120 is an even multiple
; (read interval) of 30.
; Ladislav Mecir, Gregg Irwin
round: func [
{Rounds numeric value with refinements for what kind of rounding
you want performed, how many decimal places to round to, etc.}
value [number! money! time!] {The value to round}
/up {Round away from 0}
/floor {Round towards the next more negative digit}
/ceiling {Round towards the next more positive digit}
/truncate {Remaining digits are unchanged. (a.k.a. down)}
/places {The number of decimal places to keep}
pl [integer!]
/to-interval {Round to the nearest multiple of interval}
interval [number! money! time!]
/local
factor
][
;-- places and to-interval are redundant. E.g.:
; places 2 = to-interval .01
; to-interval is more flexible so I may dump places.
;-- This sets factor in one line, under 80 chars, but is it clearer?
;factor: either places [10 ** (- pl)][either to-interval
[interval][1]]
factor: either places [
10 ** (negate pl)
] [
either to-interval [interval] [1]
]
;-- We may set truncate, floor, or ceiling in this 'if block.
if not any [up floor ceiling truncate] [
;-- Default rounding is even. Should we take the specified
; decimal places into account when rounding? We do at the
; moment.
either (abs value // factor) <> (.5 * factor) [
value: (.5 * factor) + value
return value - mod/euclid value factor
] [
;-- If we get here, it means we're rounding off exactly
; .5 (at the final decimal position that is).
either even? value [
truncate: true
] [
either negative? value [floor: true][ceiling: true]
]
]
]
if up [either negative? value [floor: true][ceiling: true]]
if truncate [return value - (value // factor)]
if floor [return value - mod/euclid value factor]
if ceiling [return value + mod/euclid (negate value) factor]
]

### [9/17] from: tomc:darkwing:uoregon at: 12-Jan-2002 19:07

a small point, the test below should be
if 1.0e-10 > absolute (a - b)
unless your sure a > b
On Wed, 9 Jan 2002, Joel Neely wrote:
...

### [10/17] from: joel:neely:fedex at: 12-Jan-2002 16:15

Hi, Tom,
Tom Conlin wrote:

> a small point, the test below should be
>
> if 1.0e-10 > absolute (a - b)
>
> unless your sure a > b
>

You're ABSOLUTEly correct! Thanks!
-jn-
--
A tool will not automatically fulfill your goals. You have to use
it.
-- John Stevens
joel)FIX)PUNCTUATION)dot)neely)at)fedex)dot)com

### [11/17] from: lmecir:mbox:vol:cz at: 13-Jan-2002 20:12

If I understood correctly, the original problem was, how to perform some
simple operations (addition in this case, but it can be generalized to the
multiplication or other operations) to get "accurate" results in the case
where the results should be numbers with two decimal digits. If you perform
enough normal additions for binary represented two-decimal digit numbers,
the error can be of any size.
There are two ways how to get "accurate" results in this case, i.e. not
allow the error to grow. The first one is, that integer arithmetic is always
accurate (no error). If you use integer numbers (the amounts are in cents
rather than in dollars), the results will always be accurate. This may be a
little bit inconvenient sometimes. The second possible solution is to use
rounding. Instead of normal add use rounded add like:
rounded-add: func [a [number!] b [number!]] [
round/places a + b 2
]
This way the error is not allowed to grow for any number of additions and if
you compare two rounded numbers, they will be equal exactly when they should
be.
Cheers
Ladislav

### [12/17] from: rebol665:ifrance at: 13-Jan-2002 23:13

Hi Ladislav,
How come there is no round function in my rebol ?
Patrick

### [13/17] from: lmecir:mbox:vol:cz at: 14-Jan-2002 12:46

Hi Pat,
<<Patrick>>
Hi Ladislav,
How come there is no round function in my rebol ?
<</Patrick>>
Excuse moi, the ROUND function you can find e.g. at
http://www.sweb.cz/LMecir/rounding.r
A bientot
L

### [14/17] from: rebol665:ifrance at: 14-Jan-2002 13:09

Bonjour Ladislav
Merci beaucoup, and I can see that your french is very good.
Patrick

### [15/17] from: moeller_thorsten:gmx at: 14-Jan-2002 12:47

Hi to all, responding to my question,
i really appreciated your explanations and samples you gave. Thanks for
this.
I tried using Ladislav's round-function, but it seems that it doesn't help
me with my problem in a way i need.
i learned from your responses that the result of 1E-2 from a substraction of
1,99 and 1,98

>> 1,99 - 1,98

== 1E-2
comes from the binary handling of this decimals an is the same as 0,01 and
will behave so in further operations. OK.
Is it possible to display the value of 0,01 correctly, without checking each
result and providing a corresponding string like >> if c = 0,01 [print
0,01
]??
(same for values lower than 0,01 like 0,001)
Thorsten

### [16/17] from: joel:neely:fedex at: 14-Jan-2002 9:37

Hi, Thorsten,
Thorsten Moeller wrote:

> i learned from your responses that the result of 1E-2 from a
> substraction of 1,99 and 1,98

<<quoted lines omitted: 7>>

> ?? (same for values lower than 0,01 like 0,001)
>From the point of view of scientific notation, 1E-2 and 0.01
*ARE* both correct representations of the same value. If what
you're saying is that you want a "commercial" presentation of
that value, you can certainly do something resembling this
as-decimal-string: func [x [decimal!] /local r t] [
r: 0.01 * to-integer 100 * x + 0.5
t: join mold to-integer r "."
r: r - to-integer r
loop 2 [
r: r * 10.0
append t mold to-integer r
r: r - to-integer r
]
t
]
so that

>> 1.99 - 1.98

== 1E-2

>> as-decimal-string 1.99 - 1.98

== "0.01"

>> as-decimal-string 1.0 / 3.0

== "0.33"

>> as-decimal-string 2.0 / 3.0

== "0.67"
WARNING: This is a quick-and-dirty sketch to show the kind of
calculations involved. It is neither general (multiple precisions)
nor complete (e.g. doesn't handle negative values) nor maximally
efficient!
However, before belaboring the above code, is this the kind of
result you're looking for?
-jn-

### [17/17] from: moeller_thorsten:gmx at: 14-Jan-2002 20:02

Hi Joel,
that is exactly what i was looking for.
Thanks for spending your time with this.
Thorsten
----------------------------------------------------------
Few can foresee whither their road will lead them,
till they come to it's end.

Notes

- Quoted lines have been omitted from some messages.

View the message alone to see the lines that have been omitted