Re: print() precision

From: Wei Huang <huangwei_at_nyahnyahspammersnyahnyah>
Date: Wed Jul 31 2013 - 10:34:12 MDT

Fan,

If you print a out, you'll see it will be truncated, as there is now way
a (as a float) to store that much digits (accuracy).

When you convert a (float) to b (double), only the part of a, which could be
represented by machine, is copied from a to b.

With NCL as a scripting language,

a = 0.4071155678

is first read into NCL as string, and then converted to a as float.

but line like:

c = 0.4071155678d

where the number is read in as string, but with "d" at the end,
it is convert to c as double.

and then a is stored in ncl as float (of 4-bytes),
and c is stored in ncl as double (of 8-bytes).

So, later, when a, or c is printed, they appear in different accuracy,
because they are in different type.

Other types have same character, find more about data types in NCL at:
http://www.ncl.ucar.edu/Document/Manuals/Ref_Manual/NclDataTypes.shtml#BasicNumericTypes

NCL has lots of type conversion functions:
http://www.ncl.ucar.edu/Document/Functions/type_convert.shtml

Wei

huangwei@ucar.edu
VETS/CISL
National Center for Atmospheric Research
P.O. Box 3000 (1850 Table Mesa Dr.)
Boulder, CO 80307-3000 USA
(303) 497-8924

On Jul 31, 2013, at 10:14 AM, Fan Fang <Fan.Fang@nasa.gov> wrote:

> Thanks Wei and Dennis.
>
> It works. But if the variable is pre-assigned without 'd' at the end,
> would 'todouble()' somehow do it? I have
>
> begin
> a = 0.4071155678
> b = todouble(a)
> print(b)
> end
>
> and I got something not quite precise:
>
> Copyright (C) 1995-2010 - All Rights Reserved
> University Corporation for Atmospheric Research
> NCAR Command Language Version 5.2.1
> The use of this software is governed by a License Agreement.
> See http://www.ncl.ucar.edu/ for more details.
>
>
> Variable: b
> Type: double
> Total Size: 8 bytes
> 1 values
> Number of Dimensions: 1
> Dimensions and sizes: [1]
> Coordinates:
> (0) 0.4071155786514282
>
>
>
>
> On 07/31/2013 12:07 PM, Dennis Shea wrote:
>> xf = 0.4071155678
>> xd = 0.4071155678d
>>
>> print("xf="+xf +" : "+sprintf("%13.6f", xf))
>> print("xd="+xd +" : "+sprintf("%13.10f", xd))
>>
>>
>> ==============
>>
>> (0) xf=0.407116 : 0.407116
>> (0) xd=0.407116 : 0.4071155678
>>
>>
>> On 7/31/13 9:57 AM, Wei Huang wrote:
>>> Fan,
>>>
>>> Try
>>>
>>> print(0.407115567890123456789d)
>>>
>>> Wei
>>>
>>> huangwei@ucar.edu
>>> VETS/CISL
>>> National Center for Atmospheric Research
>>> P.O. Box 3000 (1850 Table Mesa Dr.)
>>> Boulder, CO 80307-3000 USA
>>> (303) 497-8924
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Jul 31, 2013, at 9:41 AM, Fan Fang <Fan.Fang@nasa.gov> wrote:
>>>
>>>> It appear print() only does floats. IS there an easy way to print out
>>>> just what is fed?
>>>>
>>>> -Fan
>>>>
>>>> begin
>>>> print(0.4071155678)
>>>> end
>>>>
>>>> Copyright (C) 1995-2009 - All Rights Reserved
>>>> University Corporation for Atmospheric Research
>>>> NCAR Command Language Version 5.1.1
>>>> The use of this software is governed by a License Agreement.
>>>> See http://www.ncl.ucar.edu/ for more details.
>>>> (0) 0.4071156
>>>> _______________________________________________
>>>> ncl-talk mailing list
>>>> List instructions, subscriber options, unsubscribe:
>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>
>>> _______________________________________________
>>> ncl-talk mailing list
>>> List instructions, subscriber options, unsubscribe:
>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>
>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Wed Jul 31 10:34:36 2013

This archive was generated by hypermail 2.1.8 : Thu Aug 01 2013 - 15:55:03 MDT