Derrick,
The short answer is yes, you have answered your own question. ;-)
You may know that equality testing for floating point is a tricky thing
due to roundoff error. When writing robust code, a common method is to
do a bracket comparison low and high. For example, to compare A vs. B,
one might actually check for 0.99A < B < 1.01B.
For designated "special values" such as _FillValue, deliberately chosen
to be far away in magnitude from all reasonable values, a threshold
offset of 1% as in my example seems reasonable to me. I seem to recall
a case of some real netcdf processing software that uses this same
offset of 1%.
So yes, I think that this could cause your missing value problem. It
seems that the trouble in your case actually starts with the regridding
program, which is apparently blending intended missing values with valid
data. This is A Bad Thing, which would give you numeric values and
problems exactly as you describe. I suggest you go after that first. HTH.
--Dave A.
CDC/NOAA/CIRES
Derrick Snowden wrote:
> Hi all,
>
> I'm trying to plot a map of correlation between two fields which I generated
> using esccr. I usually insert the line below in my plotting scripts to get a
> feel for the range of the variables in the map.
>
> res@cnInfoLabelOn = True
> res@cnInfoLabelString = "min = $ZMN$, max = $ZMX$"
>
> Recently, I generated some plots with values larger than one and smaller than
> minus one. I'm using time series that have 108 points in them and am only
> looking at lags out to two or three so I don't anticipate any strange
> statistical problems. In the plots I set the contour min/max/cint
> explicitly. If I repeat the calculation and insert a line to mask values
> greater than one...
>
> ccr = mask(ccr,ccr.gt.1.0,False)
>
> The resultant plot looks, to my eye, identical. Both variables have the
> _FillValue attribute set. But, occasionally I notice trouble with fill
> values in files generated by different applications or originally stored as
> doubles but coerced to floats. Could a few outliers in the data which are
> near but not equal to a very large fill value cause this behavior? (i.e.
> _FillValue = -9.99e34 and a data point has a value -9.9892342392e34.) I
> suppose this could be the result of using linint2 to regrid the variables
> prior to the correlation calculation. This is ocean model data and those
> *$^@ coastlines make things difficult sometimes.
>
> I suppose I've answered my own question but if anyone has comments....
>
> Best,
> Derrick
>
>
_______________________________________________
ncl-talk mailing list
ncl-talk@ucar.edu
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
This archive was generated by hypermail 2b29 : Tue Feb 22 2005 - 09:00:43 MST