Re: Variable input error: what am I doing wrong?

From: Gerald Creager <gerry.creager_at_nyahnyahspammersnyahnyah>
Date: Fri Oct 07 2011 - 16:29:41 MDT

And another question: Despite claiming a fill-value for SSH is -9999,
they've actually populated either fill, or missing, with 999900. I've
added:
model4Ssh@__FillValue = 999900

but it doesn't seem to do much good. I've also tried to limit the range
of my output thus:
res@cnMinLevelValF = -100 ; Minimum SSH variation from MSL
res@cnMaxLevelValF = 80 ; Maximum SSH above MSL

For reference...
ncl 92> print(min(model4Ssh))
(0) -57.33852
ncl 93> print(max(model4Ssh))
(0) 999900

Thanks!
gerry

On 10/07/2011 04:47 PM, David Brown wrote:
> Hi Gerry,
>
> Your problem stems from the fact that although the file claims to be CF-1.0 compliant, it is not quite. The dimensions are named "lat" and "lon" but the intended coordinate variables are named "latitude" and "longitude". The CF convention is that a variable is a (1D) coordinate variable only if it has the same name as the dimension. Doing a printVarSummary on the data variable you get this:
>
> Variable: model4Ssh
> Type: float
> Total Size: 41445756 bytes
> 10361439 values
> Number of Dimensions: 3
> Dimensions and sizes: [time | 99] x [lat | 261] x [lon | 401]
> Coordinates:
> time: [734776.5.. 0]
> Number Of Attributes: 5
> _FillValue : 999900
> units : m
> __FillValue : -9999
> long_name : sea-surface height
> positive : up
>
> Notice the dimensions are named but no coordinates are associated with lat and lon. The fix is simply to add these two lines after you read in model4SSh
>
> model4Ssh&lat = model4Lat
> model4Ssh&lon = model4Lon
>
> -dave
>
> On Oct 7, 2011, at 1:59 PM, Gerald Creager wrote:
>
>> Adam, I'm back....
>>
>> We're working on a project with all sorts of purported NetCDF files I'm
>> getting to sort through. Just as an idea, the one I'm on now uses Matlab
>> datenum for its time variable format; that's a solved problem.
>>
>> It does have 1d lat/lon data as evidenced by the data from ncl_filedump
>> shown below:
>> [gerry@wagoneer Step3]$ ncl_filedump MITGOM_STEP-3_FCAST-2.nc
>> Copyright (C) 1995-2011 - All Rights Reserved
>> University Corporation for Atmospheric Research
>> NCAR Command Language Version 6.0.0
>> The use of this software is governed by a License Agreement.
>> See http://www.ncl.ucar.edu/ for more details.
>>
>> Variable: f
>> Type: file
>> filename: MITGOM_STEP-3_FCAST-2
>> path: MITGOM_STEP-3_FCAST-2.nc
>> file global attributes:
>> title : MITgcm GoM three-month forecast for GOMEX-PPP STEP-3:2011
>> Model : MITgcm
>> Filename :
>> /net/wave3/ganeshgopal/WAVE5_DATA/GOM_4DVAR_2011_RUNS/run_23jul2011/res_nc/MITGOM_STEP-3_FCAST-2.nc
>> Model_Resolution : 1/10 degree
>> Hgrid : Arakawa-C Grid
>> Stdgrid : Interpolated to standard grid : [262E-282E, 18N-31N] :
>> 0.05 deg resolution : ( IM=401, JM=201 )
>> Vgrid : Z - level (40 levels)
>> Stdlevel : Interpolated to standard level: 22 levels ( 0, -20,
>> -40, -60, -80, -100, -150, -200, -250, -300, -400, -500, -600, -700,
>> -800, -900, -1000, -1500, -2000, -2500, -3000, -3500 m )
>> Institution : Scripps Institution of Oceanography
>> Conventions : CF-1.0
>> Contact : Bruce Cornuelle (bdc@ucsd.edu), Ganesh Gopal
>> (ggopalak@ucsd.edu)
>> dimensions:
>> lon = 401
>> lat = 261
>> zlev = 22
>> time = 99 // unlimited
>> variables:
>> float longitude ( lon )
>> units : degrees_east
>> long_name : longitude at cell-center
>>
>> float latitude ( lat )
>> units : degrees_north
>> long_name : latitude at cell-center
>>
>> float zlev ( zlev )
>> units : m
>> long_name : depth at cell-center
>> negative : down
>>
>> float time ( time )
>> units : matlab datenum
>> long_name : date
>>
>> float sea_floor_depth_below_sea_level ( lat, lon )
>> units : m
>> __FillValue : -9999
>> long_name : depth
>> positive : down
>>
>> float sea_surface_height_above_sea_level ( time, lat, lon )
>> units : m
>> __FillValue : -9999
>> long_name : sea-surface height
>> positive : up
>>
>> float eastward_sea_water_velocity_at_bottom ( time, lat, lon )
>> units : m s-1
>> __FillValue : -9999
>> long_name : bottom u
>>
>> float northward_sea_water_velocity_at_bottom ( time, lat, lon )
>> units : m s-1
>> __FillValue : -9999
>> long_name : bottom v
>>
>> float temp_at_bottom ( time, lat, lon )
>> units : deg C
>> __FillValue : -9999
>> long_name : bottom temp
>>
>> float salinity_at_bottom ( time, lat, lon )
>> units : psu
>> __FillValue : -9999
>> long_name : bottom salinity
>>
>> float eastward_sea_water_velocity ( time, zlev, lat, lon )
>> units : m s-1
>> __FillValue : -9999
>> long_name : eastward component of velocity
>>
>> float northward_sea_water_velocity ( time, zlev, lat, lon )
>> units : m s-1
>> __FillValue : -9999
>> long_name : northward component of velocity
>>
>> float sea_water_potential_temperature ( time, zlev, lat, lon )
>> units : deg C
>> __FillValue : -9999
>> long_name : potential temperature
>>
>> float sea_water_salinity ( time, zlev, lat, lon )
>> units : psu
>> __FillValue : -9999
>> long_name : salinity
>>
>>
>> I'm getting a lat/lon error similar to what I was seeing over the last
>> couple of days with the last "compliant" file data:
>> (0) check_for_y_lat_coord: Warning: Data either does not contain a valid
>> latitude coordinate array or d
>> oesn't contain one at all.
>> (0) A valid latitude coordinate array should have a 'units' attribute
>> equal to one of the following val
>> ues:
>> (0) 'degrees_north' 'degrees-north' 'degree_north' 'degrees north'
>> 'degrees_N' 'Degrees_north' 'deg
>> ree_N' 'degreeN' 'degreesN' 'deg north'
>> (0) check_for_lon_coord: Warning: Data either does not contain a valid
>> longitude coordinate array or do
>> esn't contain one at all.
>> (0) A valid longitude coordinate array should have a 'units' attribute
>> equal to one of the following va
>> lues:
>> (0) 'degrees_east' 'degrees-east' 'degree_east' 'degrees east'
>> 'degrees_E' 'Degrees_east' 'degree_E
>> ' 'degreeE' 'degreesE' 'deg east'
>>
>>
>> I'm currently uploading MITGOM_STEP-3_FCAST-2.nc to the FTP site for
>> perusal, if y'all have time. I'm also uploading Step23-Model4.ncl.
>>
>> Both are going into 'incoming'.
>>
>> Thanks for all your support. I suspect this is another minor gotcha but
>> I'm lost.
>>
>> gerry
>>
>> On 10/06/2011 11:28 AM, Adam Phillips wrote:
>>> Hi Gerry,
>>> When you have 2D latitudes/longitudes, you need to assign the 2D lat/lon
>>> arrays as lat2d/lon2d attributes of your data array.
>>>
>>> I tried this with the data file you sent in:
>>>
>>> ncl 0> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"
>>> ncl 1> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"
>>> ncl 2> a = addfile("fcast_201009.nc","r")
>>> ncl 3> t = a->t(:,0,:,:)
>>> ncl 4> t@lat2d = a->alat
>>> ncl 5> t@lon2d = a->alon
>>> ncl 6> wks = gsn_open_wks("x11","test")
>>> ncl 7> plot = gsn_csm_contour_map(wks,t(0,:,:),False)
>>>
>>> and it worked. Let us know if you continue to have a problem plotting
>>> the data.. Adam
>>>
>>> On 10/06/2011 09:10 AM, Gerald Creager wrote:
>>>> I just tried but passive mode was refused (I've had this problem
>>>> recently here, too!). Donna or I will get a subset of one of the model
>>>> files uploaded, as well as the script shortly.
>>>>
>>>> Thanks, Gerry
>>>>
>>>>
>>>> On 10/06/2011 10:01 AM, Dennis Shea wrote:
>>>>> GC .. Send 2 or 3 sample files and a minimalistic script that isolates
>>>>> the issues you are having.
>>>>>
>>>>> ftp ftp.cgd.ucar.edu
>>>>> anonymous
>>>>> Use your email address for the password
>>>>> cd incoming
>>>>> put ...
>>>>> put ...
>>>>> :
>>>>> quit
>>>>>
>>>>> Send email to ncl-talk with the following information:
>>>>>
>>>>> The names of the files after you have successfully completed the transfer.
>>>>> We cannot look at the ftp/incoming directory for security reasons.
>>>>> The type of system you are on (uname -a).
>>>>>
>>>>> THX
>>>>> On 10/6/11 8:33 AM, Gerald Creager wrote:
>>>>>> I checked last night. We'*ARE* running v6.0.0:
>>>>>> [root@scoopdata gomexstep3]# ncl
>>>>>> Copyright (C) 1995-2011 - All Rights Reserved
>>>>>> University Corporation for Atmospheric Research
>>>>>> NCAR Command Language Version 6.0.0
>>>>>> The use of this software is governed by a License Agreement.
>>>>>> See http://www.ncl.ucar.edu/ for more details.
>>>>>>
>>>>>> but the problem persisted. Not a real problem. I'm looking for SST and
>>>>>> SSH at this point, so I snagged the top-level temperature and I'm good
>>>>>> on this.
>>>>>>
>>>>>> Now, however, another problem with the same netcdf input. The snippet of
>>>>>> code where I'm assigning variables for lat/lon:
>>>>>>
>>>>>> model2Lat=model2[:]->alat ; Degrees NORTH
>>>>>> model2Lon=model2[:]->alon ; Degrees WEST (?)
>>>>>>
>>>>>> And, a print statement to see what the vars look like:
>>>>>> print(model2Lat)
>>>>>>
>>>>>>
>>>>>> Variable: model2Lat
>>>>>> Type: float
>>>>>> Total Size: 3767796 bytes
>>>>>> 941949 values
>>>>>> Number of Dimensions: 2
>>>>>> Dimensions and sizes: [y | 2349] x [x | 401]
>>>>>> Coordinates:
>>>>>> Number Of Attributes: 3
>>>>>> long_name : latitude
>>>>>> units : deg-N
>>>>>> coordinates : alon alat
>>>>>> (0,0) 18
>>>>>> (0,1) 18
>>>>>> (0,2) 18
>>>>>> (0,3) 18
>>>>>> (0,4) 18
>>>>>> ...
>>>>>>
>>>>>> print(model2Lon)
>>>>>>
>>>>>>
>>>>>> Variable: model2Lon
>>>>>> Type: float
>>>>>> Total Size: 3767796 bytes
>>>>>> 941949 values
>>>>>> Number of Dimensions: 2
>>>>>> Dimensions and sizes: [y | 2349] x [x | 401]
>>>>>> Coordinates:
>>>>>> Number Of Attributes: 3
>>>>>> long_name : longitude
>>>>>> units : deg-W
>>>>>> coordinates : alon alat
>>>>>> (0,0) -98
>>>>>> (0,1) -97.95
>>>>>> (0,2) -97.9
>>>>>> (0,3) -97.85
>>>>>> (0,4) -97.8
>>>>>> (0,5) -97.75
>>>>>> (0,6) -97.7
>>>>>> ...
>>>>>>
>>>>>> OK, units are screwy:
>>>>>> model2Lat@units = "degrees_north"
>>>>>> model2Lon@units = "degrees_east"
>>>>>> print(model2Lat)
>>>>>>
>>>>>>
>>>>>> Variable: model2Lat
>>>>>> Type: float
>>>>>> Total Size: 3767796 bytes
>>>>>> 941949 values
>>>>>> Number of Dimensions: 2
>>>>>> Dimensions and sizes: [y | 2349] x [x | 401]
>>>>>> Coordinates:
>>>>>> Number Of Attributes: 3
>>>>>> long_name : latitude
>>>>>> units : degrees_north
>>>>>> coordinates : alon alat
>>>>>> (0,0) 18
>>>>>> (0,1) 18
>>>>>> (0,2) 18
>>>>>> (0,3) 18
>>>>>> ...
>>>>>> print(model2Lon)
>>>>>>
>>>>>>
>>>>>> Variable: model2Lon
>>>>>> Type: float
>>>>>> Total Size: 3767796 bytes
>>>>>> 941949 values
>>>>>> Number of Dimensions: 2
>>>>>> Dimensions and sizes: [y | 2349] x [x | 401]
>>>>>> Coordinates:
>>>>>> Number Of Attributes: 3
>>>>>> long_name : longitude
>>>>>> units : degrees_east
>>>>>> coordinates : alon alat
>>>>>> (0,0) -98
>>>>>> (0,1) -97.95
>>>>>> (0,2) -97.9
>>>>>> (0,3) -97.85
>>>>>> ...
>>>>>>
>>>>>> When I plot, I get a plot that isn't at the resolution I'm expecting,
>>>>>> and I get:
>>>>>> (0) check_for_y_lat_coord: Warning: Data either does not contain a valid
>>>>>> latitude coordinate array or doesn't co
>>>>>> ntain one at all.
>>>>>> (0) A valid latitude coordinate array should have a 'units' attribute
>>>>>> equal to one of the following values:
>>>>>> (0) 'degrees_north' 'degrees-north' 'degree_north' 'degrees north'
>>>>>> 'degrees_N' 'Degrees_north' 'degree_N' 'd
>>>>>> egreeN' 'degreesN' 'deg north'
>>>>>> (0) check_for_lon_coord: Warning: Data either does not contain a valid
>>>>>> longitude coordinate array or doesn't con
>>>>>> tain one at all.
>>>>>> (0) A valid longitude coordinate array should have a 'units' attribute
>>>>>> equal to one of the following values:
>>>>>> (0) 'degrees_east' 'degrees-east' 'degree_east' 'degrees east'
>>>>>> 'degrees_E' 'Degrees_east' 'degree_E' 'degree
>>>>>> E' 'degreesE' 'deg east'
>>>>>>
>>>>>> As near as I can tell, I've reassigned units in an appropriate manner.
>>>>>> I'm guessing this is is associate with gsn_csm_contour_map, but I've not
>>>>>> found where I've gone wrong.
>>>>>>
>>>>>> Thanks,
>>>>>> gerry
>>>>>>
>>>>>> On 10/05/2011 06:18 PM, Gerry Creager wrote:
>>>>>>> I'll give it a try in a little while (parental duties right now!)
>>>>>>>
>>>>>>> gc
>>>>>>>
>>>>>>> Adam Phillips wrote:
>>>>>>>> Hi Gerry,
>>>>>>>> The newest version of ncl (v6.0.0) allows array sizes larger than 2GB.
>>>>>>>> Previous versions do not. I am guessing you are using a pre-6.0.0
>>>>>>>> version and your aggregated T array is (potentially) larger than 2GB..
>>>>>>>>
>>>>>>>> If I'm off on my guess let us know.. Adam
>>>>>>>>
>>>>>>>> On 10/05/2011 04:50 PM, Gerald Creager wrote:
>>>>>>>>> Thanks, Gentlemen. I didn't realize that (but I should have)...
>>>>>>>>>
>>>>>>>>> OK, so I tried this:
>>>>>>>>> model2Sst=model2[:]->t ; potential temperature with
>>>>>>>>> coordinates : time, zz, y, x
>>>>>>>>> fatal:Aggregating variable t from file list variable model2 as
>>>>>>>>> specified
>>>>>>>>> would exceed maximum NCL variable size
>>>>>>>>>
>>>>>>>>> gerry
>>>>>>>>>
>>>>>>>>> On 10/05/2011 04:45 PM, David Brown wrote:
>>>>>>>>>> Hi Gerry,
>>>>>>>>>> When you use 'addfiles' rather than 'addfile', you get a file list
>>>>>>>>>> variable rather than a file variable. It requires a different syntax.
>>>>>>>>>> Try
>>>>>>>>>> model2Sst=model2[:]->t
>>>>>>>>>> and
>>>>>>>>>> Time = model2[:]->time
>>>>>>>>>>
>>>>>>>>>> You can say 'model2[0]' in order to "reference a file".
>>>>>>>>>> -dave
>>>>>>>>>>
>>>>>>>>>> On Oct 5, 2011, at 3:09 PM, Gerald Creager wrote:
>>>>>>>>>>
>>>>>>>>>>> I'm trying to assign values to a variable from a netcdf file
>>>>>>>>>>> (ncl_filedump output for the file is attached). A sample snippet
>>>>>>>>>>> of code is attached. Note the error... And, note that line 6
>>>>>>>>>>> does, in fact load the data into 'model2'.
>>>>>>>>>>>
>>>>>>>>>>> ncl 30> model2Sst=model2->t
>>>>>>>>>>> fatal:(model2) does not reference a filencl 0> load
>>>>>>>>>>> "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"
>>>>>>>>>>> ncl 1> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"
>>>>>>>>>>> ncl 2> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"
>>>>>>>>>>> ncl 3> load
>>>>>>>>>>> "$NCARG_ROOT/lib/ncarg/nclscripts/contrib/calendar_decode2.ncl"
>>>>>>>>>>> ncl 4>
>>>>>>>>>>> ncl 5> fnameModel2=systemfunc ("ls fcast_20100*.nc") ; files
>>>>>>>>>>> location
>>>>>>>>>>> ncl 6> model2=addfiles(fnameModel2, "r")
>>>>>>>>>>> ncl 7> type="x11"
>>>>>>>>>>> ncl 8> Time=model2->time
>>>>>>>>>>> fatal:(model2) does not reference a file
>>>>>>>>>>> fatal:Execute: Error occurred at or near line 8
>>>>>>>>>>>
>>>>>>>>>>> fatal:Execute: Error occurred at or near line 30
>>>>>>>>>>>
>>>>>>>>>>> Thanks, Gerry
>>>>>>>>>>> -
>>>>>>>>>>> Gerry Creager -- gerry.creager@tamu.edu
>>>>>>>>>>> Texas Mesonet -- AATLT, Texas A&M University
>>>>>>>>>>> Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.862.3983
>>>>>>>>>>> Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX
>>>>>>>>>>> 77843
>>>>>>>>>>> <nclfiledump-model2.txt>_______________________________________________
>>>>>>>>>>>
>>>>>>>>>>> ncl-talk mailing list
>>>>>>>>>>> List instructions, subscriber options, unsubscribe:
>>>>>>>>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>
>>
>> --
>> Gerry Creager -- gerry.creager@tamu.edu
>> Texas Mesonet -- AATLT, Texas A&M University
>> Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.862.3983
>> Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX 77843
>> _______________________________________________
>> ncl-talk mailing list
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>

-- 
Gerry Creager -- gerry.creager@tamu.edu
Texas Mesonet -- AATLT, Texas A&M University
Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.862.3983
Office: 7607 Eastmark Drive, Suite 112, College Station, TX 77840
_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Fri Oct 7 16:29:49 2011

This archive was generated by hypermail 2.1.8 : Sun Oct 09 2011 - 13:05:26 MDT