Re: unexpected behavior with floattoshort and _FillValue

From: Mary Haley <haley_at_nyahnyahspammersnyahnyah>
Date: Fri, 4 Apr 2008 08:37:36 -0600 (MDT)

Jamie,

This might be a bug, but I need to check with Dave Brown.

What's happening is that whenever you have an array with a _FillValue,
and you use one of the xxxxtoyyyy functions to convert it to another
type, it's going to try to convert the _FillValue to that type as
well. However, if the original _FillValue (1e30 in your case) is of a
size that can't be converted to the type of the new array, then the
default _FillValue for the new type (-99 for short) will be used
instead, which, as you saw, gave you unexpected results.

Furthermore, as you saw, "floattoshort" does not appear to see the
already existing _FillValue of your sdata array; hence, it is not
using this value instead of the default -99 value to compare against
the original values in your float array.

I will find out if this is a bug.

Meanwhile, as you probably already know, a fix for this is to use a
float _FillValue that can be converted to a short, so this way it
won't try to use -99 as the _FillValue.

--Mary

On Thu, 3 Apr 2008, Jamie Scott wrote:

> I have an array of floating point values that I need to convert to short
> values using floattoshort.
> The array (fdata) doesn't have any missing values, but I define the
> _FillValue to be 1.e30, for good measure.
> The short array (sdata) that is the target of the floattoshort function also
> has a _FillValue defined to be floattoshort(32767),
> as I'm trying to avoid the default short value of -99. When I coerce fdata
> to sdata using floattoshort, the values that
> should be -99 are converted to the _FillValue of 32767.
>
> Similarly, if I create the same floating point array (fdata2), but do not
> define an _FillValue (use the default -999), the coercion to
> sdata2 using floattoshort works as I expected -- floating point value of
> -99.31335 becomes -99 -- not _FillValue.
>
> Since I'm defining my short array to use 32767 as the _FillValue (instead of
> the default -99), I expected this would avoid the problem of floating point
> values
> between -99 and -100 becoming _FillValue when they pass through
> floattoshort. I was further surprised that using an _FillValue other than
> the
> default on the input floating point array made a difference to floattoshort.
>
> I don't understand why floattoshort gives different results dependent on the
> value of _FillValue in the input floating point array. Does this make
> sense to anyone?
>
> (I'm using ncl version 5.0.0 on OSX 10.4)
>
> Here's a simple demonstation:
> ;;;;;;;;;;;;;;;;;;;;;;;;ncl code;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
> fdata=new((/8/),"float",1.e30)
> sdata=new((/8/),"short",floattoshort(32767))
> fdata=
> (/-101.1139,-100.7233,-100.3479,-99.99084,-99.99,-99.31335,-98.99597,-98
> .6908/)
> print(fdata(5))
> sdata=floattoshort(fdata)
> print(sdata(5))
> print(ismissing(sdata(5)))
>
> fdata2=new((/8/),"float") ;;don't define a missing value for float array
> sdata2=new((/8/),"short",floattoshort(32767))
> fdata2=
> (/-101.1139,-100.7233,-100.3479,-99.99084,-99.99,-99.31335,-98.99597,-98
> .6908/)
> print(fdata2(5))
> sdata2=floattoshort(fdata2)
> print(sdata2(5))
> print(ismissing(sdata2(5)))
> ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
>
> ncl printout::
>
>
> Variable: fdata (subsection)
> Type: float
> Total Size: 4 bytes
> 1 values
> Number of Dimensions: 1
> Dimensions and sizes: [1]
> Coordinates:
> Number Of Attributes: 1
> _FillValue : 1e+30
> (0) -99.31335
>
>
> Variable: sdata (subsection)
> Type: short
> Total Size: 2 bytes
> 1 values
> Number of Dimensions: 1
> Dimensions and sizes: [1]
> Coordinates:
> Number Of Attributes: 1
> _FillValue : 32767
> (0) 32767
> (0) True
>
>
> Variable: fdata2 (subsection)
> Type: float
> Total Size: 4 bytes
> 1 values
> Number of Dimensions: 1
> Dimensions and sizes: [1]
> Coordinates:
> Number Of Attributes: 1
> _FillValue : -999
> (0) -99.31335
>
>
> Variable: sdata2 (subsection)
> Type: short
> Total Size: 2 bytes
> 1 values
> Number of Dimensions: 1
> Dimensions and sizes: [1]
> Coordinates:
> Number Of Attributes: 1
> _FillValue : 32767
> (0) -99
> (0) False
>
> ---------------------
>
> Jamie Scott
> NOAA/ESRL/PSD
> james.d.scott_at_noaa.gov
>
>
_______________________________________________
ncl-talk mailing list
ncl-talk_at_ucar.edu
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Fri Apr 04 2008 - 08:37:36 MDT

This archive was generated by hypermail 2.2.0 : Fri Apr 04 2008 - 08:43:08 MDT