Re: Floattoint truncating issue.

Yi,

The precision of numbers of NCL type float is only 6 decimal digits.
  With 8 digit numbers you are exceeding the precision of type
float, and getting roundoff errors. Use type int or type double
instead, or else handle the year, month, and day as separate numbers.

--Dave A.
CU/CIRES Climate Diagnostics Center (CDC)
NOAA/ESRL/PSD, Climate Analysis Branch (CAB)
http://www.cdc.noaa.gov/

Wang, Yi wrote:
> Hi Folks:
>
> For example, I have a time series from 19961010 to 19961020.
>
> When reading in as float, I get this (print-out):
>
> (0) 1.996101e+07
> (1) 1.996101e+07
> (2) 1.996101e+07
> (3) 1.996101e+07
> (4) 1.996101e+07
> (5) 1.996102e+07
> (6) 1.996102e+07
> (7) 1.996102e+07
> (8) 1.996102e+07
> (9) 1.996102e+07
> (10) 1.996102e+07
>
>
> To do a "floattoint" on above series, I get this (print-out):
>
> (0) 19961010
> (1) 19961012 "" (should be 19961011)""
> (2) 19961012
> (3) 19961012 ""(should be 19961013)""
> (4) 19961014
> (5) 19961016
> (6) 19961016
> (7) 19961016
> (8) 19961018
> (9) 19961020
> (10) 19961020
>
> The actual date should go from 19961010 to 19961020. How do I fix this
> floattoint trancating issue?
>
> Thanks,
>
> Yi
> _______________________________________________
> ncl-talk mailing list
> ncl-talk_at_ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk

_______________________________________________
ncl-talk mailing list
ncl-talk_at_ucar.edu
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Tue Oct 16 2007 - 14:48:28 MDT