Mailing List Archive: 49091 messages
  • Home
  • Script library
  • AltME Archive
  • Mailing list
  • Articles Index
  • Site search
 

[REBOL] Re: decimal woes

From: lmecir::mbox::vol::cz at: 18-Jul-2003 17:23

Hi Max, you can use e.g.: do http://www.fm.vslib.cz/~ladislav/rebol/rounding.r time: 0 loop 50 [ time: round/to time + 0.02 0.01 probe time ] -L ----- Original Message ----- Hi all, I am having fun converting SEISMIC data (sampled at 5 thousands of a second :-) within a 30MB file. I must dump all of this in a 3d animation package (maya) and its working... except for one point.. decimal datatype is a little unstable, and maybe someone can tell me why this happens: if you do: time: 0 loop 50 [ time: time + 0.02 probe time ] ask "" at the end you will have 0.92 0.940000000000001 0.960000000000001 0.980000000000001 but if you then do 0.92 + 0.02 you get 0.94 the floating point error gets worse and worse (and eventually diminishes a little!!!) there are only two signifiant digits in all calculus, I am wondering if it could be possible for the floating point algorythms to be a little more stable... I had to an ugly integer calculation workaround divided by 100 to get the clean decimals... -max