[REBOL] Re: decimal woes
From: lmecir::mbox::vol::cz at: 18-Jul-2003 17:23
Hi Max,
you can use e.g.:
do http://www.fm.vslib.cz/~ladislav/rebol/rounding.r
time: 0
loop 50 [
time: round/to time + 0.02 0.01
probe time
]
-L
----- Original Message -----
Hi all,
I am having fun converting SEISMIC data (sampled at 5 thousands of a second
:-) within a 30MB file.
I must dump all of this in a 3d animation package (maya) and its working...
except for one point..
decimal datatype is a little unstable, and maybe someone can tell me why
this happens:
if you do:
time: 0
loop 50 [
time: time + 0.02
probe time
]
ask ""
at the end you will have
0.92
0.940000000000001
0.960000000000001
0.980000000000001
but if you then do 0.92 + 0.02
you get 0.94
the floating point error gets worse and worse (and eventually diminishes a
little!!!)
there are only two signifiant digits in all calculus, I am wondering if it
could be possible for the floating point algorythms to be a little more
stable...
I had to an ugly integer calculation workaround divided by 100 to get the
clean decimals...
-max