# Re: print() precision

From: Fan Fang <Fan.Fang_at_nyahnyahspammersnyahnyah>
Date: Thu Aug 01 2013 - 09:02:27 MDT

Dave,

I suppose I understand your explanation - it's up to the programmer to
read in and store a decimal number as a string and convert it to a
decimal number before using it. If it is read and stored as a decimal
number, unless the type is known and stored, only its precisions can be
worked on.

-Fan

On 07/31/2013 09:54 PM, Dave Allured - NOAA Affiliate wrote:
> Fan,
>
> Hmmm. We explained in different ways that many decimal numbers can
> not be stored exactly in floats and doubles, there is a loss of
> precision on input. This is basic math, and has nothing to do with
> NCL in particular. This is true even for absurdly simple numbers like
> 0.1.
>
> We also showed you several ways of displaying variables with increased
> precision, including more than necessary precision so you could be
> sure to get all SIGNIFICANT decimal digits.
>
> So what more are you looking for in a new function? I think if you
> show another very simple example, with the EXACT string output that
> you would like to see, we can reach a better understanding.
>
> --Dave
>
> On Wed, Jul 31, 2013 at 5:32 PM, Fang, Fan (GSFC-610.2)[ADNET SYSTEMS
> INC] <fan.fang-1@nasa.gov> wrote:
>> Thanks Wei and everyone for investigating this.
>>
>> However not under all situations do we know the value of a double or want to format it. It'd be great if NCL has a function that prints exactly what the float or double value a variable holds, like treating it as a string.
>>
>> -Fan
>> ________________________________________
>> From: Wei Huang [huangwei@ucar.edu]
>> Sent: Wednesday, July 31, 2013 3:39 PM
>> To: Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]
>> Cc: Dennis Shea; ncl-talk@ucar.edu
>> Subject: Re: print() precision
>>
>> Fan,
>>
>> NCL uses "%g" in its C program to print float or double.
>> This will give 6 digit when print, or convert to string.
>>
>> The program below can clearly demonstrate that.
>>
>> If you need more accurate print, please use the sprintf function
>> (also shown in program below).
>>
>> Wei
>>
>> ---------------
>> a = 1.0d
>> b = 11.11d
>>
>> do n = 0, 20
>> print("No. " + n + ": a(%g) = " + a + ", a(%32.16f) = " + sprintf("%32.16f", a))
>>
>> a = a * b
>> end do
>>
>> a = 1.0d
>> b = 0.1111d
>>
>> do n = 0, 20
>> print("No. " + n + ": a(%g) = " + a + ", a(%32.16f) = " + sprintf("%32.16f", a))
>>
>> a = a * b
>> end do
>>
>>
>> huangwei@ucar.edu
>> VETS/CISL
>> National Center for Atmospheric Research
>> P.O. Box 3000 (1850 Table Mesa Dr.)
>> Boulder, CO 80307-3000 USA
>> (303) 497-8924
>>
>>
>> On Jul 31, 2013, at 10:14 AM, Fan Fang <Fan.Fang@nasa.gov> wrote:
>>
>>> Thanks Wei and Dennis.
>>>
>>> It works. But if the variable is pre-assigned without 'd' at the end,
>>> would 'todouble()' somehow do it? I have
>>>
>>> begin
>>> a = 0.4071155678
>>> b = todouble(a)
>>> print(b)
>>> end
>>>
>>> and I got something not quite precise:
>>>
>>> University Corporation for Atmospheric Research
>>> NCAR Command Language Version 5.2.1
>>> The use of this software is governed by a License Agreement.
>>> See http://www.ncl.ucar.edu/ for more details.
>>>
>>>
>>> Variable: b
>>> Type: double
>>> Total Size: 8 bytes
>>> 1 values
>>> Number of Dimensions: 1
>>> Dimensions and sizes: [1]
>>> Coordinates:
>>> (0) 0.4071155786514282
>>>
>>>
>>> On 07/31/2013 12:07 PM, Dennis Shea wrote:
>>>> xf = 0.4071155678
>>>> xd = 0.4071155678d
>>>>
>>>> print("xf="+xf +" : "+sprintf("%13.6f", xf))
>>>> print("xd="+xd +" : "+sprintf("%13.10f", xd))
>>>>
>>>> ==============
>>>>
>>>> (0) xf=0.407116 : 0.407116
>>>> (0) xd=0.407116 : 0.4071155678
>>>>
>>>>
>>>> On 7/31/13 9:57 AM, Wei Huang wrote:
>>>>> Fan,
>>>>>
>>>>> Try
>>>>>
>>>>> print(0.407115567890123456789d)
>>>>>
>>>>> Wei
>>>>>
>>>>> huangwei@ucar.edu
>>>>> VETS/CISL
>>>>> National Center for Atmospheric Research
>>>>> P.O. Box 3000 (1850 Table Mesa Dr.)
>>>>> Boulder, CO 80307-3000 USA
>>>>> (303) 497-8924
>>>>>
>>>>>
>>>>> On Jul 31, 2013, at 9:41 AM, Fan Fang <Fan.Fang@nasa.gov> wrote:
>>>>>
>>>>>> It appear print() only does floats. IS there an easy way to print out
>>>>>> just what is fed?
>>>>>>
>>>>>> -Fan
>>>>>>
>>>>>> begin
>>>>>> print(0.4071155678)
>>>>>> end
>>>>>>
>>>>>> (0) 0.4071156

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Thu Aug 1 09:02:33 2013

This archive was generated by hypermail 2.1.8 : Thu Aug 01 2013 - 15:55:03 MDT