Re: stringtofloat

Hi Yongzuo,

The float type only has a precision of between 6 and 7 decimal places.
In order to represent 10 digits precisely you need to use double.

However, you should also be aware that the default printed value for
float types rounds to 6 digits of precision. The value stored
internally and used for calculations is not likely to be exactly equal
to the printed value. You can print out a more precise version of the
internal value using the 'sprintf' routine:

ncl 10> print(sprintf("%10f",f))
(0) 2008090112.000000

It's not exactly the number you started with, but it's closer than it
looks using a simple print command. However, you still need to use
double to get the necessary precision to represent a date accurately.

  -dave

On Jun 9, 2010, at 5:41 PM, Li, Yongzuo wrote:

> Hi,
>
> I have a script to test stringtofloat
>
> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"
> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"
>
> begin
>
> s = "2008090100"
> f = stringtofloat(s)
> i = stringtoint(s)
>
> print("string= "+s)
> print("float= "+f)
> print("int= "+i)
>
> end
>
> The output for float is wrong
>
> (0) string= 2008091000
> (0) float= 2.00809e+09
> (0) int= 2008090100
>
> Yongzuo
>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Wed Jun 9 18:23:30 2010