Mailing List Archive: 49091 messages

# strange results with decimal!

### [1/17] from: moeller_thorsten::gmx::de at: 10-Jan-2002 11:47

Hi rebolers, i have a script that gives to produce an output when there is a difference between two values. Therefore i read a textfile, convert some fields to decimal values and add them together to one or another sum. if a specific condition comes true they are are compared should be 0 (checked manually with a small amount). For some reason i get results like "-1.3476653466456E-12". Does anybody understand this? Any suggestions will be appreciated!! Thorsten -- -------------------------------------------------------------------------------------- Few can foresee whither their road will lead them, till they come to it's end.

### [2/17] from: joel:neely:fedex at: 9-Jan-2002 23:59

Hi, Thorsten, Thorsten Moeller wrote:
> Hi rebolers, > i have a script that gives to produce an output when there is a
<<quoted lines omitted: 5>>
> Does anybody understand this? > Any suggestions will be appreciated!!
I strongly suspect you're falling victim to binary round-off. In general it is A Bad Idea to perform floating-point (decimal!) calculations and then expect to get an exact equality; rather you should check for the difference between two values being very small. For example, instead of a: ...lots of decimal! calculation... b: ...more decimal! calculation or literal... if a = b ... you should try a: ...as above... b: ...as above... if 1.0e-10 > a - b ... This is a well-known phenomenon in numerical analysis; to see why it's the case, imagine performing the following decimal calculation on a calculator with a fixed number of digits (or with pencil and paper using only e.g., 5 digits): a: 1.0 / 3.0 b: 1.0 / 3.0 c: 1.0 / 3.0 print a + b + c If you use exactly 5 digits of precision, you'll get a = 0.3333 b = 0.3333 c = 0.3333 a + b + c = 0.9999 since 1/3 does not have a terminating fractional expansion. The only divisors which produce terminating fractions are those containing only powers of 2 and/or 5, since those are the only divisors of 10. Since nothing but 2 divides 2, the only terminating binary fractions are those with divisors of powers of two. Since we use decimal externally, the conversion to binary and subsequent calculation are likely to produce the phenomenon. The reason why most desk calculators will perform the above computation and return 1.0 is that they keep additional hidden digits beyond what is displayed, and then round to the display size. If we had kept 6 digits and rounded to 5 digits for display, we'd have gotten a result of 0.99999 and rounded that to the (displayable) 1.0000 HTH! -jn- -- Shouting at the ground doesn't enable it to hear any better. -- Henry Warwick joel^dot^FIX^PUNCTUATION^neely^at^fedex^dot^com

### [3/17] from: moeller_thorsten:gmx at: 10-Jan-2002 13:39

Hi Joel, thanks for your reply. If devision or multiplication is involvolved i could follow your samples. Perhaps i should be more detailed, as i am only using addition and substraction. File sheme: 15888;some text;1.000,50;1.000,50;some text 15889;some text;50,49;50,48;some text As the first line contains german values, i have to parse out the "." and the convert the field to a decimal like 1.000,50 to 1000,50. After that i add the values to a sum with a foreach loop. the result is: sum-a: 1050,99 sum-b: 1050,98 Now i want the difference to be displayed. print sum-a - sum-b The result is, even with this example, "9.99999999999091E-3" instead of 0,01. If you just type in the lines with the sums and the print statement you can follow the example. Is that the same problem as you described? Thorsten

### [4/17] from: rebol665:ifrance at: 10-Jan-2002 14:37

Hi, Decimal are store in IEEE format which is basically made of a binary mantissa 0,xxxxxxxx and an exponent. Converting decimal so binary leads to minor error. To understand this try to convert 0,75 and 0,80 in binary. 0,75 is 0,11 in base two ( 0,1 + 0,01 or 0,5 + 0,25) To understand this, you must remember that the comma moves to the right to multiply by two ( 1 is 0,1 times two ). So 0,1 binary is 0,5 decimal. You can use this to verify that 0,75 decimal is 0,11 binary. 0,11 binary with two right moves gives 11 binary which is 3 decimal ( O,75 * 2 * 2 ). If you try to do this with 0,80 you will find it never ends round because you have to builds 0,8 decimal by addition of number like 0,5 0,25 0,125 0,0625 0,03125 etc. So 0,8 in binary is like 1/3 in decimal Patrick

### [5/17] from: philb:upnaway at: 10-Jan-2002 22:43

Hi Thorsten, I would think that the problem is the way computers hold decimals. As a decimal is held as a binary representaion of the decimal in a finite number of digits (usually 16,32 or 64) in most cases it cannot hold a decimal exactly and some rounding occurs. We dont know how internally Rebol hold decimal numbers but as its C based it may well hold them in base 2 decimal form. For example 9.3 = 9 * 10 + 3 * 0.1 has the binary form 1001.01001 (recurring) meaning 1*2^3 + 0*2^2 + 0*2^1 + 1*2^0 + 0*2^-1 + 1*2^-2 + 0*2^-3 + 0*2^-4 + 1*2^-5 As this is a recurring binary decimal some rounding will occur so the decimal number is not held exactly. Consider the following (typed at the console)
>> 0.33333333333333 + 0.333333333333333 + 0.333333333333333
== 0.999999999999996 This indicates that the number 0.33333333333333 was truncated. I would suggest that one or both of numbers in your example was truncated and gave you the unexpected result. Cheers Phil === Original Message === Hi Joel, thanks for your reply. If devision or multiplication is involvolved i could follow your samples. Perhaps i should be more detailed, as i am only using addition and substraction. File sheme: 15888;some text;1.000,50;1.000,50;some text 15889;some text;50,49;50,48;some text As the first line contains german values, i have to parse out the "." and the convert the field to a decimal like 1.000,50 to 1000,50. After that i add the values to a sum with a foreach loop. the result is: sum-a: 1050,99 sum-b: 1050,98 Now i want the difference to be displayed. print sum-a - sum-b The result is, even with this example, "9.99999999999091E-3" instead of 0,01. If you just type in the lines with the sums and the print statement you can follow the example. Is that the same problem as you described? Thorsten

### [6/17] from: joel:neely:fedex at: 10-Jan-2002 10:10

Hi, Thorsten, Thorsten Moeller wrote:
> sum-a: 1050,99 > sum-b: 1050,98
<<quoted lines omitted: 4>>
> If you just type in the lines with the sums and the print > statement you can follow the example.
Or even
>> 1050.99 - 1050.98
== 9.99999999999091E-3
> Is that the same problem as you described? >
Yes, because converting the string "1050.99" to a number actually does require division; it's equal to 105099 / 100 !
>> (105099 / 100) - (105098 / 100)
== 9.99999999999091E-3 Just as 1/3 in decimal is 0.333333333333333333333333333333333333... remember that 1/5 in binary is 0.00110011001100110011001100110011... and therefore 1/10 is 0.00011001100110011001100110011001100110011... The only two-digit decimal fractions that have exact representations as binary fractions are 0.25, 0.50, and 0.75; any other number of hundredths will encounter round-off, even if you're only taking a string representation of it into decimal. -jn-

### [7/17] from: holger:rebol at: 10-Jan-2002 8:59

On Thu, Jan 10, 2002 at 10:43:14PM +0800, [philb--upnaway--com] wrote:
> Hi Thorsten, > > I would think that the problem is the way computers hold decimals. > As a decimal is held as a binary representaion of the decimal in a finite number of digits (usually 16,32 or 64) in most cases it cannot hold a decimal exactly and some rounding occurs. > > We dont know how internally Rebol hold decimal numbers but as its C based it may well hold them in base 2 decimal form.
REBOL uses whatever format the platform it is running on uses. Typically this is IEEE double precision, i.e. base-2. -- Holger Kruse [holger--rebol--com]

### [8/17] from: greggirwin:mindspring at: 10-Jan-2002 13:43

Hi Thorsten, As Joel pointed out, you can compare decimal numbers using a small delta to account for rounding errors. Another approach is to manually round the numbers and compare the rounded results. Below is a ROUND function (and a supporting MOD function), based on Ladislav's work, if you want to go that route. --Gregg ; Ladislav Mecir, Gregg Irwin (minor adjustment) mod: func [ {Compute a remainder.} value1 [number! money! time!] {The dividend} value2 [number! money! time!] {The divisor} /euclid {Compute a non-negative remainder such that: a = qb + r and r < b} /local r ] [ either euclid [ either negative? r: value1 // value2 [r + abs value2] [r] ;-- Alternate implementation ;value1 // value2 + (value2: abs value2) // value2 ][ value1 // value2 ] ] ;-- Note: to-interval does mod-like rounding. If the interval you ; specify is not evenly divisble into your base, the result ; may not be what you expect. E.g. round/to-interval 133 30 ; will round to 120, not 130, because 120 is an even multiple ; (read interval) of 30. ; Ladislav Mecir, Gregg Irwin round: func [ {Rounds numeric value with refinements for what kind of rounding you want performed, how many decimal places to round to, etc.} value [number! money! time!] {The value to round} /up {Round away from 0} /floor {Round towards the next more negative digit} /ceiling {Round towards the next more positive digit} /truncate {Remaining digits are unchanged. (a.k.a. down)} /places {The number of decimal places to keep} pl [integer!] /to-interval {Round to the nearest multiple of interval} interval [number! money! time!] /local factor ][ ;-- places and to-interval are redundant. E.g.: ; places 2 = to-interval .01 ; to-interval is more flexible so I may dump places. ;-- This sets factor in one line, under 80 chars, but is it clearer? ;factor: either places [10 ** (- pl)][either to-interval [interval]] factor: either places [ 10 ** (negate pl) ] [ either to-interval [interval]  ] ;-- We may set truncate, floor, or ceiling in this 'if block. if not any [up floor ceiling truncate] [ ;-- Default rounding is even. Should we take the specified ; decimal places into account when rounding? We do at the ; moment. either (abs value // factor) <> (.5 * factor) [ value: (.5 * factor) + value return value - mod/euclid value factor ] [ ;-- If we get here, it means we're rounding off exactly ; .5 (at the final decimal position that is). either even? value [ truncate: true ] [ either negative? value [floor: true][ceiling: true] ] ] ] if up [either negative? value [floor: true][ceiling: true]] if truncate [return value - (value // factor)] if floor [return value - mod/euclid value factor] if ceiling [return value + mod/euclid (negate value) factor] ]

### [9/17] from: tomc:darkwing:uoregon at: 12-Jan-2002 19:07

a small point, the test below should be if 1.0e-10 > absolute (a - b) unless your sure a > b On Wed, 9 Jan 2002, Joel Neely wrote: ...

### [10/17] from: joel:neely:fedex at: 12-Jan-2002 16:15

Hi, Tom, Tom Conlin wrote:
> a small point, the test below should be > > if 1.0e-10 > absolute (a - b) > > unless your sure a > b >
You're ABSOLUTEly correct! Thanks! -jn- -- A tool will not automatically fulfill your goals. You have to use it. -- John Stevens joel)FIX)PUNCTUATION)dot)neely)at)fedex)dot)com

### [11/17] from: lmecir:mbox:vol:cz at: 13-Jan-2002 20:12

If I understood correctly, the original problem was, how to perform some simple operations (addition in this case, but it can be generalized to the multiplication or other operations) to get "accurate" results in the case where the results should be numbers with two decimal digits. If you perform enough normal additions for binary represented two-decimal digit numbers, the error can be of any size. There are two ways how to get "accurate" results in this case, i.e. not allow the error to grow. The first one is, that integer arithmetic is always accurate (no error). If you use integer numbers (the amounts are in cents rather than in dollars), the results will always be accurate. This may be a little bit inconvenient sometimes. The second possible solution is to use rounding. Instead of normal add use rounded add like: rounded-add: func [a [number!] b [number!]] [ round/places a + b 2 ] This way the error is not allowed to grow for any number of additions and if you compare two rounded numbers, they will be equal exactly when they should be. Cheers Ladislav

### [12/17] from: rebol665:ifrance at: 13-Jan-2002 23:13

Hi Ladislav, How come there is no round function in my rebol ? Patrick

### [13/17] from: lmecir:mbox:vol:cz at: 14-Jan-2002 12:46

Hi Pat, <<Patrick>> Hi Ladislav, How come there is no round function in my rebol ? <</Patrick>> Excuse moi, the ROUND function you can find e.g. at http://www.sweb.cz/LMecir/rounding.r A bientot L

### [14/17] from: rebol665:ifrance at: 14-Jan-2002 13:09

Bonjour Ladislav Merci beaucoup, and I can see that your french is very good. Patrick

### [15/17] from: moeller_thorsten:gmx at: 14-Jan-2002 12:47

Hi to all, responding to my question, i really appreciated your explanations and samples you gave. Thanks for this. I tried using Ladislav's round-function, but it seems that it doesn't help me with my problem in a way i need. i learned from your responses that the result of 1E-2 from a substraction of 1,99 and 1,98
>> 1,99 - 1,98
== 1E-2 comes from the binary handling of this decimals an is the same as 0,01 and will behave so in further operations. OK. Is it possible to display the value of 0,01 correctly, without checking each result and providing a corresponding string like >> if c = 0,01 [print 0,01 ]?? (same for values lower than 0,01 like 0,001) Thorsten

### [16/17] from: joel:neely:fedex at: 14-Jan-2002 9:37

Hi, Thorsten, Thorsten Moeller wrote:
> i learned from your responses that the result of 1E-2 from a > substraction of 1,99 and 1,98
<<quoted lines omitted: 7>>
> ?? (same for values lower than 0,01 like 0,001) >From the point of view of scientific notation, 1E-2 and 0.01
*ARE* both correct representations of the same value. If what you're saying is that you want a "commercial" presentation of that value, you can certainly do something resembling this as-decimal-string: func [x [decimal!] /local r t] [ r: 0.01 * to-integer 100 * x + 0.5 t: join mold to-integer r "." r: r - to-integer r loop 2 [ r: r * 10.0 append t mold to-integer r r: r - to-integer r ] t ] so that
>> 1.99 - 1.98
== 1E-2
>> as-decimal-string 1.99 - 1.98
== "0.01"
>> as-decimal-string 1.0 / 3.0
== "0.33"
>> as-decimal-string 2.0 / 3.0
== "0.67" WARNING: This is a quick-and-dirty sketch to show the kind of calculations involved. It is neither general (multiple precisions) nor complete (e.g. doesn't handle negative values) nor maximally efficient! However, before belaboring the above code, is this the kind of result you're looking for? -jn-

### [17/17] from: moeller_thorsten:gmx at: 14-Jan-2002 20:02

Hi Joel, that is exactly what i was looking for. Thanks for spending your time with this. Thorsten ---------------------------------------------------------- Few can foresee whither their road will lead them, till they come to it's end.

Notes
• Quoted lines have been omitted from some messages.
View the message alone to see the lines that have been omitted