Re: Why there are missing values in NARR data?

From: Dennis Shea <shea_at_nyahnyahspammersnyahnyah>
Date: Sun, 10 May 2009 17:31:56 -0600

[1] Missing NARR values

http://www.ncl.ucar.edu/Applications/narr.shtml
See the last paragraph that points you toward a power point
presentation on NARR data. Specifically, it states:

---
Important note: in the corners of the images on this page, you may 
notice some small slivers of missing data. The original NARR Eta-12 
model grid is regridded to standard NCEP grid 221 for public 
distribution, Lambert conformal conic, which you see here.
There is a slight mismatch between the chosen grid boundaries, resulting 
in the slivers of missing data. This is discussed on pages 39, 41, 42 of 
this Powerpoint summary from NCEP (2005). Page 42 is the picture that 
speaks a thousand words:
     http://www.emc.ncep.noaa.gov/mmb/rreanl/narr.ppt
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
[2] data of type 'short'
http://www.ncl.ucar.edu/Document/Functions/Contributed/short2flt.shtml
See Example 1
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
[3[ missing_value and _FillValue
_FillValue is recognized by NCL, the netCDF operators and
thr CF netCDF convention. It is an official Unidata netCDF
reserved attribute name.
missing_value is from the COARDS netCDF convention.
Technically, there is nothing wrong with specifying both.
However, using two different values is not  common.
One possible practical use: Consider SSTs from ships.
Over land the grid points are set to _FillValue.
Over the ocen there could be grid points that 'could'
havd data but do not. These could be zassigned
missing_value.
You can count the number of _FillValue via
      nFill = num (ismissing(snowc) )
      nMiss = num (snowc .eq. snowc_at_missing_vale)
      print("nFill="+nFill+"   nMiss="+nMiss)
Good luck
Tao.Zhang_at_noaa.gov wrote:
> Xianglei,
> 
>  Thanks for your comments. It seems that valid_range = -32766s, 22765s,
> which corresponds to 0 ~1 if scale_factor and add_offset are considered.
> Why missing values are 32766, but FillValue = -32767 ? see below. What
> does this mean?
> 
>  Thanks,
>  Tao
> 
>    snowc:unpacked_valid_range = 0.f, 1.f ;
>>>                 snowc:precision = 4s ;
>>>                 snowc:actual_range = 0.f, 1.f ;
>>>                 snowc:add_offset = 3.2766f ;
>>>                 snowc:scale_factor = 0.0001f ;
>>>                 snowc:missing_value = 32766s ;
>>>                 snowc:valid_range = -32766s, 22765s ;
>>>                 snowc:_FillValue = -32767s ;
> 
> 
> 
> 
> ----- Original Message -----
> From: Xianglei Huang <xianglei_at_umich.edu>
> Date: Sunday, May 10, 2009 11:13 am
> Subject: Re: Why there are missing values in NARR data?
> 
>> Hi Alligator,
>>
>> 	Not aware you are NCL user, hoho.
>>
>> 	There could be missing data in model output, which means model 
>> unable 
>> to give a value there. For example, there should be missing value 
>> of 
>> temperature at 1000mb over Tibet Plateau because the surface 
>> altitude is 
>> already very high. Your snow cover case, I suspect, is due to 
>> incompatible and conflicting results of data assimilation, so in 
>> posteriori fixing, they rather put missing value. Or that place 
>> with 
>> snow and rain same time, cannot tell a real snow cover, et al.
>>
>> 	Xianglei
>>
>> Tao.Zhang_at_noaa.gov wrote:
>>> Hi,
>>>
>>>   I am trying to regrid NARR (North American Regional Reanlysis) 
>> data> (netcdf format, see point 2 below) into 2X2 regular netcdf 
>> grid. I find
>>> that the regrided 2X2 netcdf data have values of 6.5532 for snow 
>> cover,> it is obviously wrong. I print out the original NARR snow 
>> cover (netcdf
>>> format) data (the code is listed in point 1 below), and the message
>>> below shows that NARR raw data have values of 6.5532 already. As 
>> NARR> data is model ouput, there is no reason that the data should 
>> have> missing values. Actually, the snow cover should range from 0 
>> to 1.
>>> (snowc:unpacked_valid_range = 0.f, 1.f ;
>>>                 snowc:precision = 4s ;
>>>                 snowc:actual_range = 0.f, 1.f ;)
>>>
>>>
>>> Therefore, I am sure that 6.5532 comes from the error calculation 
>> using> the missing values (6.5532=32766*0.0001+3.2766).
>>>   But I don't understand why this occurs (there should be no missing
>>> values for NARR data as it is model output)? How to get around this
>>> missing values? 
>>>
>>>   Your suggestions are much appreciated.
>>>
>>>    Thanks,
>>>    Tao
>>>
>>>
>>>   
>>>
>>> ----------------------------
>>> Variable: x
>>> Type: float
>>> Total Size: 134568816 bytes
>>>             33642204 values
>>> Number of Dimensions: 3
>>> Dimensions and sizes:   [348] x [277] x [349]
>>> Coordinates: 
>>> Number Of Attributes: 1
>>>   _FillValue :  -32767
>>> (0)      min=0   max=6.5532
>>>
>>>
>>> Variable: x (subsection)
>>> Type: float
>>> Total Size: 386692 bytes
>>>             96673 values
>>> Number of Dimensions: 2
>>> Dimensions and sizes:   [277] x [349]
>>> Coordinates: 
>>> Number Of Attributes: 1
>>>   _FillValue :  -32767
>>> (0,0)    0
>>> (0,1)    0
>>> (0,2)    0
>>>
>>> ......
>>> (45,343)         0
>>> (45,344)         0
>>> (45,345)         0
>>> (45,346)        6.5532
>>> (45,347)        6.5532
>>> (45,348)        6.5532
>>> (46,0)   0
>>> (46,1)   0
>>> (46,2)   0
>>> (46,3)   0
>>> (46,4)   0
>>> (46,5)   0
>>> (46,6)   0
>>> --------------------------
>>>
>>>
>>>
>>>
>>>
>>>   
>>> (1) NCL Code
>>> ;**************************************************************
>>> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"
>>> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"
>>> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"
>>>
>>> begin
>>> ;*******************************************
>>> ; open file and read in data
>>> ;*******************************************
>>>   diri    = "../raw_nc/"                              ; input 
>> directory>   f       = addfile (diri+"snowc.mon.mean.nc", "r")
>>>                                               ; convert short=>float
>>> ;  x       = short2flt( f->snowc(:,:,:) )     ; (time,lat,lon)
>>> ;
>>>
>>>   scale_factor = f->snowc_at_scale_factor
>>>   add_offset = f->snowc_at_add_offset
>>>   x = scale_factor * (f->snowc) + add_offset
>>>
>>>   time   = f->time
>>>   lat2d   = f->lat                            ; coordinates
>>>   lon2d   = f->lon
>>>   dimlc   = dimsizes(lat2d)                   ; dimension sizes
>>>   nlat    = dimlc(0)
>>>   mlon    = dimlc(1)
>>>   ntime   = dimsizes(time)
>>> ;
>>>   print("lat2d: min="+min(lat2d)+"   max="+max(lat2d))
>>>
>>>   printVarSummary(x)
>>>   print(" min="+min(x)+"   max="+max(x))
>>>
>>>   print(x(0,:,:))
>>> end
>>>     
>>>
>>>
>>> (2) information about NARR (North American Regional Reanlysis) data
>>>
>>> netcdf snowc.mon.mean {
>>> dimensions:
>>>         time = UNLIMITED ; // (348 currently)
>>>         y = 277 ;
>>>         x = 349 ;
>>>         nbnds = 2 ;
>>> variables:
>>>         float lat(y, x) ;
>>>                 lat:long_name = "latitude coordinate" ;
>>>                 lat:units = "degrees_north" ;
>>>                 lat:axis = "Y" ;
>>>                 lat:coordinate_defines = "point" ;
>>>                 lat:standard_name = "latitude" ;
>>>         float lon(y, x) ;
>>>                 lon:units = "degrees_east" ;
>>>                 lon:long_name = "longitude coordinate" ;
>>>                 lon:axis = "X" ;
>>>                 lon:coordinate_defines = "point" ;
>>>                 lon:standard_name = "longitude" ;
>>>         float x(x) ;
>>>                 x:long_name = "eastward distance from southwest 
>> corner> of domain in projection coordinates" ;
>>>                 x:units = "m" ;
>>>                 x:standard_name = "projection_x_coordinate" ;
>>>         float y(y) ;
>>>                 y:long_name = "northward distance from southwest 
>> corner> of domain in projection coordinates" ;
>>>                 y:units = "m" ;
>>>                 y:standard_name = "projection_y_coordinate" ;
>>>         int Lambert_Conformal ;
>>>                 Lambert_Conformal:grid_mapping_name =
>>> "lambert_conformal_conic" ;
>>>                 Lambert_Conformal:standard_parallel = 50., 50. ;
>>>                 Lambert_Conformal:longitude_of_central_meridian = 
>> -107. ;
>>>                 Lambert_Conformal:latitude_of_projection_origin = 
>> 50. ;
>>>                 Lambert_Conformal:false_easting = 5632642.22547 ;
>>>                 Lambert_Conformal:false_northing = 4612545.65137 ;
>>>
>>>          short snowc(time, y, x) ;
>>>                 snowc:units = "1" ;
>>>                 snowc:long_name = "Monthly Snow Cover at Surface" ;
>>>                 snowc:unpacked_valid_range = 0.f, 1.f ;
>>>                 snowc:precision = 4s ;
>>>                 snowc:actual_range = 0.f, 1.f ;
>>>                 snowc:add_offset = 3.2766f ;
>>>                 snowc:scale_factor = 0.0001f ;
>>>                 snowc:missing_value = 32766s ;
>>>                 snowc:valid_range = -32766s, 22765s ;
>>>                 snowc:_FillValue = -32767s ;
>>>                 snowc:GRIB_name = "SNOWC" ;
>>>                 snowc:GRIB_id = 238 ;
>>>                 snowc:var_desc = "snow cover" ;
>>>                 snowc:standard_name = "" ;
>>>                 snowc:level_desc = "Surface" ;
>>>                 snowc:dataset = "NARR Monthly Averages" ;
>>>                 snowc:statistic = "Mean" ;
>>>                 snowc:parent_stat = "Individual Obs" ;
>>>                 snowc:grid_mapping = "Lambert_Conformal" ;
>>>                 snowc:coordinates = "lat lon" ;
>>>                 snowc:cell_methods = "time: mean (monthly from daily
>>> values)" ;
>>>
>>> _______________________________________________
>>> ncl-talk mailing list
>>> List instructions, subscriber options, unsubscribe:
>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>
>>>
>>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Sun May 10 2009 - 17:31:56 MDT

This archive was generated by hypermail 2.2.0 : Mon May 11 2009 - 09:43:46 MDT