NCL default missing values

Hi NCL users,

We are tentatively planning to modify NCL's default fill values in
order to put them further
outside the range of "normal" data sets and computations. This will
apply to variables defined in NCL
as well as to variables defined in GRIB files as presented by NCL.
The motivating factor for this
change is that recently we have encountered GRIB data where the data
range includes the value used
for float type fill value (-999), leading to a situation where a few
valid values are treated as missing.
Of course this has always been a possibility but has not been
encountered in practice, or at least,
it has not jumped out as a problem in our tests or reported as a bug
by any users until now. HDF and
NetCDF file variables will not be affected because the NCL
representation of the variable only contains
a _FillValue attribute if it is defined in the file.

We only want to do this once so we would like to get input on the
best values to use, as well as
feedback on possible problems to existing code.

Our plan is to change the default fill values for float and double
values to the value 1.0e20.

The byte, character, and and string missing values (255, inttochar
(0), and "missing") will not change.

Note that the long type changes size between 32 and 64 bit machines.
On 64 bit machines the long type can hold much bigger
values, but I think we want to ensure that the default long fill
value will always be the same.
Therefore the default long fill value and the default integer fill
value will probably be equal.

For int, long and short we could follow the example of NetCDF and use

long and int:
-2147283647 (maximum acceptable INT_MIN according to IEEE std.
1003.1, 2004 edition)
short
-32767 (maximum acceptable SHRT_MIN)

but it could certainly be argued that something with all '9''s such
as +/-999999999 (for long and integer) and (+/-9999) for short
would be easier to type, remember, and recognize when visually
scanning the contents of a variable.

Other suggestions and arguments pro or con any alternatives are welcome.

FYI, we are planning soon to add more integer types including
especially a 64-bit integer type, but also specifically unsigned
versions of all the types.
Our decisions concerning fill values for the existing types will be
extrapolated to come up with fill values for these new types.

  -dave

_______________________________________________
ncl-talk mailing list
ncl-talk_at_ucar.edu
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Fri Apr 18 2008 - 15:35:41 MDT