NCL is 'netCDF convention aware'. The nc file you are reading violate
aspects of the COARDS and CF conventions.
[1]
The COARDS and CF netCDF conventions and the Unidata recommended
"Best Practices" state the the "missing_value" attribute should be the
same type
as the variable with which it is associated.
http://www.unidata.ucar.edu/software/netcdf/docs/BestPractices.html#Missing%20Data%20Values
short qnet(time, lat, lon) ; <==== short
qnet:missing_value = 32766. ; <==== double
NCL is telling you that there is a type mismatch.
The missing_value should be of type short. It is not.
print( typeof(qnet_at_missing_value)) ====> "double"
[2]
The "missing_value" attribute is "deprecated" under the CF netCDF
convention.
NCL only recognizes the "_FillValue" attribute to indicate missing or
undefined data.
When NCL looks at a variable with only a "missing_value" and no "_FillValue"
attribute , it creates a 'virtual' _FillValue attribute and asociates
it with the variable.
However, when NCL sees a type mismatch, it does not do this. Hence, the
warning message:
"not adding virtual _FillValue attribute"
[3]
NCL's "setfileoption" allows the user to turn off NCL's default behavior
http://www.ncl.ucar.edu/Document/Functions/Built-in/setfileoption.shtml
setfileoption ("nc", "/MissingToFillValue"/, False) ; turn off
default behavior
[4]
The units for the lat/lon coordinate variables are not CF/COARDS conformant.
"degrees" is not good. Change to recognized units.
See attached script ....
Good Luck
D
Leslie Hartten wrote:
> Hello,
>
> For the first time, I'm reading in gridded (instead of station) data
> that's in netCDF (instead of ascii) format, and it's not going smoothly.
> The files I'm trying to work with hold daily data from the OAFlux
> dataset, downloaded directly from WHOI. ncdump shows the following:
>
> netcdf qnet_1985 {
> dimensions:
> time = UNLIMITED ; // (365 currently)
> lat = 180 ;
> lon = 360 ;
> variables:
> float lon(lon) ;
> lon:long_name = "longitude" ;
> lon:units = "degrees" ;
> lon:range = "0.5 to 359.5" ;
> float lat(lat) ;
> lat:long_name = "latitude" ;
> lat:units = "degrees" ;
> lat:range = "89.5S to 89.5N" ;
> int time(time) ;
> time:units = "day" ;
> time:comment = "Year 1985" ;
> short qnet(time, lat, lon) ;
> qnet:comments = "daily mean net surface heat flux,
> positive downward" ;
> qnet:units = "W/m^2" ;
> qnet:scale_factor = 0.1 ;
> qnet:missing_value = 32766. ;
>
> netcdf ts_oaflux_1985 {
> dimensions:
> time = UNLIMITED ; // (365 currently)
> lat = 180 ;
> lon = 360 ;
> variables:
> float lon(lon) ;
> lon:long_name = "longitude" ;
> lon:units = "degrees" ;
> lon:range = "0.5 to 359.5" ;
> float lat(lat) ;
> lat:long_name = "latitude" ;
> lat:units = "degrees" ;
> lat:range = "89.5S to 89.5N" ;
> int time(time) ;
> time:units = "day" ;
> time:comment = "Year 1985" ;
> short tmpsf(time, lat, lon) ;
> tmpsf:comments = "daily mean sea surface temperature" ;
> tmpsf:units = "degree C" ;
> tmpsf:scale_factor = 0.01 ;
> tmpsf:missing_value = 32766. ;
> short err(time, lat, lon) ;
> err:comments = "estimated error of analysed sea surface
> temperature" ;
> err:units = "degree C" ;
> err:scale_factor = 0.01 ;
> err:missing_value = 32766. ;
> -----
>
> My NCL code (derived loosely from godas_3.ncl, since I can't get the
> multi-year part to work) is as follows:
> ----- -----
> netheat_fils =
> "/Volumes/Disk/papeeha/database/OAFlux/dailyWHOI/netheat_1985-2004/qnet_1985.nc"
> sst_fils =
> "/Volumes/Disk/papeeha/database/OAFlux/dailyWHOI/turbulence_1985-2006/ts_oaflux_1985.nc"
>
> f1 = addfiles (netheat_fils, "r")
> print(netheat_fils)
> f2 = addfiles (sst_fils, "r")
> print(sst_fils)
>
> ; read/unpack Qnet & SST from all yearly files
> qnet = short2flt( f1[:]->qnet(:,{-40:40},{235:335}) )
> printVarSummary (qnet)
> sst = short2flt( f2[:]->tmpsf(:,{-40:40},{235:335}) )
> printVarSummary (sst)
> ----- -----
>
> I'm running version = ncl.ppc.5.1.1. A full trace of the output is
> appended at the end of this email, but the key problem is the
> "warning:NetOpenFile" messages and the fact that missing_value is being
> reset:
> ----- ----- -----
> warning:NetOpenFile: MissingToFillValue option set True, but
> missing_value attribute and data variable (qnet) types differ: not
> adding virtual _FillValue attribute
>
> warning:NetOpenFile: MissingToFillValue option set True, but
> missing_value attribute and data variable (tmpsf) types differ: not
> adding virtual _FillValue attribute
>
> warning:NetOpenFile: MissingToFillValue option set True, but
> missing_value attribute and data variable (err) types differ: not adding
> virtual _FillValue attribute
> ----- ----- -----
>
> Why am I getting that NetOpenFile warning and missing values set to
> 1e+20 (see full trace for details)?
>
> Cheers,
> Leslie
> ----- ----- ----- -----
> Full trace of NCL job:
>
> version = ncl.ppc.5.1.1
> Copyright (C) 1995-2009 - All Rights Reserved
> University Corporation for Atmospheric Research
> NCAR Command Language Version 5.1.1
> The use of this software is governed by a License Agreement.
> See http://www.ncl.ucar.edu/ for more details.
> warning:NetOpenFile: MissingToFillValue option set True, but
> missing_value attribute and data variable (qnet) types differ: not
> adding virtual _FillValue attribute
>
>
>
> Variable: netheat_fils
> Type: string
> Total Size: 4 bytes
> 1 values
> Number of Dimensions: 1
> Dimensions and sizes: [1]
> Coordinates:
> (0)
> /Volumes/Disk/papeeha/database/OAFlux/dailyWHOI/netheat_1985-2004/qnet_1985.nc
> warning:NetOpenFile: MissingToFillValue option set True, but
> missing_value attribute and data variable (tmpsf) types differ: not
> adding virtual _FillValue attribute
>
> warning:NetOpenFile: MissingToFillValue option set True, but
> missing_value attribute and data variable (err) types differ: not adding
> virtual _FillValue attribute
>
>
>
> Variable: sst_fils
> Type: string
> Total Size: 4 bytes
> 1 values
> Number of Dimensions: 1
> Dimensions and sizes: [1]
> Coordinates:
> (0)
> /Volumes/Disk/papeeha/database/OAFlux/dailyWHOI/turbulence_1985-2006/ts_oaflux_1985.nc
>
>
> Variable: qnet
> Type: float
> Total Size: 11680000 bytes
> 2920000 values
> Number of Dimensions: 3
> Dimensions and sizes: [time | 365] x [lat | 80] x [lon | 100]
> Coordinates:
> time: [1..365]
> lat: [-39.5..39.5]
> lon: [235.5..334.5]
> Number Of Attributes: 4
> _FillValue : 1e+20
> comments : daily mean net surface heat flux, positive downward
> units : W/m^2
> missing_value : 1e+20
>
>
> Variable: sst
> Type: float
> Total Size: 11680000 bytes
> 2920000 values
> Number of Dimensions: 3
> Dimensions and sizes: [time | 365] x [lat | 80] x [lon | 100]
> Coordinates:
> time: [1..365]
> lat: [-39.5..39.5]
> lon: [235.5..334.5]
> Number Of Attributes: 4
> _FillValue : 1e+20
> comments : daily mean sea surface temperature
> units : degree C
> missing_value : 1e+20
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
-- ====================================================== Dennis J. Shea tel: 303-497-1361 | P.O. Box 3000 fax: 303-497-1333 | Climate Analysis Section | Climate & Global Dynamics Div. | National Center for Atmospheric Research | Boulder, CO 80307 | USA email: shea 'at' ucar.edu | ======================================================
load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"
var = "qnet"
latS = -40
latN = 40
lonL = 235
lonR = 335
diri = "/project/cas/shea/WHOI_FLUX/"
fili = systemfunc("cd "+diri+" ; ls "+var+"*.nc")
nfil = dimsizes(fili)
;print(fili)
setfileoption ("nc", "MissingToFillValue", False) ; turn off default behavior
f = addfiles (diri+fili, "r")
xShort = f[:]->$var$(:,{latS:latN},{lonL:lonR})
; fix non standard attributes
xShort@_FillValue = doubletoshort(xShort_at_missing_value)
delete(xShort_at_missing_value) ; delete badly typed attribute
xShort_at_missing_value = xShort@_FillValue ; assign correct type
; fix non-udunits for lat and lon
xShort&lat_at_units = "degrees_north"
xShort&lon_at_units = "degrees_east"
printVarSummary(xShort)
x = short2flt( xShort )
printVarSummary( x )
printMinMax( x, True )
_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Fri Aug 28 2009 - 11:45:05 MDT
This archive was generated by hypermail 2.2.0 : Wed Sep 02 2009 - 12:24:39 MDT