Re: irregular values from fspan

From: David Brown <dbrown_at_nyahnyahspammersnyahnyah>
Date: Tue Nov 26 2013 - 12:44:48 MST

Hi Kathryn,
You are just running into an issue of the print precision. By default, NCL prints float values with a precision of 6 significant digits, causing in your case truncation of the decimal part of the value.
If you change the print statement to:

print(sprintf("%10.2f",seq(0:6))) ;print test of sequence from span

You will see the full precision of your values:

(0) 4229667.00
(1) 4229665.50
(2) 4229664.00
(3) 4229662.50
(4) 4229661.00
(5) 4229659.50
(6) 4229658.00

Note however that you are very near the maximum possible float precision, so in some case you may see float point rounding inaccuracies. You could switch to double precision if you have trouble.
 -dave

On Nov 26, 2013, at 12:25 PM, "Bormann, Kathryn J (3246-Affiliate)" <Kathryn.J.Bormann@jpl.nasa.gov> wrote:

> y1 = 4229667.
> y2 = 4179327.
> printVarSummary(y1)
>
> Variable: y1
> Type: float
> Total Size: 4 bytes
> 1 values
> Number of Dimensions: 1
> Dimensions and sizes: [1]
> Coordinates:
> ncl 25> printVarSummary(y2)
>
> Variable: y2
> Type: float
> Total Size: 4 bytes
> 1 values
> Number of Dimensions: 1
> Dimensions and sizes: [1]
> Coordinates:
> n = 33560
> seq = fspan( y1, y2, n)
> print(seq(0:6)) ;print test of sequence from fspan
>
>
> Variable: seq (subsection)
> Type: float
> Total Size: 28 bytes
> 7 values
> Number of Dimensions: 1
> Dimensions and sizes: [7]
> Coordinates:
> (0) 4229667
> (1) 4229666
> (2) 4229664
> (3) 4229662
> (4) 4229661
> (5) 4229660
> (6) 4229658
> ncl 29> spacing = (y2 - y1)/n
> ncl 30> print(spacing)
>
> ;---expected spacing
> Variable: spacing
> Type: float
> Total Size: 4 bytes
> 1 values
> Number of Dimensions: 1
> Dimensions and sizes: [1]
> Coordinates:
> (0) -1.5

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Tue Nov 26 12:44:59 2013

This archive was generated by hypermail 2.1.8 : Wed Dec 04 2013 - 20:42:38 MST