Re: About numeric precision

Andy,

Let us look at your code:

 a = new(1,double)
 a = 19800103
 print(a)

 a = new(1,double)
 a = 19800103.
 print(a)

Here, you have defined a as double,
but the right handle sight is read based on its suffix.

where a = 19800103, the right handle side is read in as integer, and then converted to double.
where a = 19800103., the right handle side is read in as float, and then coerced to double
(the is why the print give you a different number).

As you have found out, to make a have the expected behavior,
you need to write a = 19800103.d, here d is the suffix for double.

Where for float you can do: b = 123 or b = 123i for integer,
and c = 123., for float (where it has ".", but does not have d as suffix).

For asciiread, it should based on the type you want to read.

Hope this help,

Wei

huangwei@ucar.edu
VETS/CISL
National Center for Atmospheric Research
P.O. Box 3000 (1850 Table Mesa Dr.)
Boulder, CO 80307-3000 USA
(303) 497-8924

On Aug 3, 2012, at 12:35 PM, Cheung wrote:

>
> Thanks!
>
> And if I read data from an ascii file, and I use the double array as
> xxx = new((/a,b,/),double)
> xxx = asciiread(x,x,x)
>
> will that happen again?
> or it depends on my data?
>
>
>
> At 2012-08-04 02:29:24,"Andrew Mai" <mai@ucar.edu> wrote:
> >On 08/03/2012 12:24 PM, Cheung wrote:
> >
> >> a = new(1,double)
> >> a = 19800103. (plus a point)
> >> print(a)
> >> a will be 19800104
> >
> >You have converted a 32-bit float to a 64-bit float. 32-bit floats only have about 7 decimal places of accuracy.
> >
> >If, instead, you did:
> >
> >a = 19800103.d0
> >print(a)
> >
> >you would get 19800103 as you expect.
> >
> >Andy
>
>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Fri Aug 3 14:17:15 2012