Re: Variable input error: what am I doing wrong?

From: Gerald Creager <gerry.creager_at_nyahnyahspammersnyahnyah>
Date: Fri Oct 07 2011 - 13:59:49 MDT

Adam, I'm back....

We're working on a project with all sorts of purported NetCDF files I'm
getting to sort through. Just as an idea, the one I'm on now uses Matlab
datenum for its time variable format; that's a solved problem.

It does have 1d lat/lon data as evidenced by the data from ncl_filedump
shown below:
[gerry@wagoneer Step3]$ ncl_filedump MITGOM_STEP-3_FCAST-2.nc
  Copyright (C) 1995-2011 - All Rights Reserved
  University Corporation for Atmospheric Research
  NCAR Command Language Version 6.0.0
  The use of this software is governed by a License Agreement.
  See http://www.ncl.ucar.edu/ for more details.

Variable: f
Type: file
filename: MITGOM_STEP-3_FCAST-2
path: MITGOM_STEP-3_FCAST-2.nc
    file global attributes:
       title : MITgcm GoM three-month forecast for GOMEX-PPP STEP-3:2011
       Model : MITgcm
       Filename :
/net/wave3/ganeshgopal/WAVE5_DATA/GOM_4DVAR_2011_RUNS/run_23jul2011/res_nc/MITGOM_STEP-3_FCAST-2.nc
       Model_Resolution : 1/10 degree
       Hgrid : Arakawa-C Grid
       Stdgrid : Interpolated to standard grid : [262E-282E, 18N-31N] :
0.05 deg resolution : ( IM=401, JM=201 )
       Vgrid : Z - level (40 levels)
       Stdlevel : Interpolated to standard level: 22 levels ( 0, -20,
-40, -60, -80, -100, -150, -200, -250, -300, -400, -500, -600, -700,
-800, -900, -1000, -1500, -2000, -2500, -3000, -3500 m )
       Institution : Scripps Institution of Oceanography
       Conventions : CF-1.0
       Contact : Bruce Cornuelle (bdc@ucsd.edu), Ganesh Gopal
(ggopalak@ucsd.edu)
    dimensions:
       lon = 401
       lat = 261
       zlev = 22
       time = 99 // unlimited
    variables:
       float longitude ( lon )
          units : degrees_east
          long_name : longitude at cell-center

       float latitude ( lat )
          units : degrees_north
          long_name : latitude at cell-center

       float zlev ( zlev )
          units : m
          long_name : depth at cell-center
          negative : down

       float time ( time )
          units : matlab datenum
          long_name : date

       float sea_floor_depth_below_sea_level ( lat, lon )
          units : m
          __FillValue : -9999
          long_name : depth
          positive : down

       float sea_surface_height_above_sea_level ( time, lat, lon )
          units : m
          __FillValue : -9999
          long_name : sea-surface height
          positive : up

       float eastward_sea_water_velocity_at_bottom ( time, lat, lon )
          units : m s-1
          __FillValue : -9999
          long_name : bottom u

       float northward_sea_water_velocity_at_bottom ( time, lat, lon )
          units : m s-1
          __FillValue : -9999
          long_name : bottom v

       float temp_at_bottom ( time, lat, lon )
          units : deg C
          __FillValue : -9999
          long_name : bottom temp

       float salinity_at_bottom ( time, lat, lon )
          units : psu
          __FillValue : -9999
          long_name : bottom salinity

       float eastward_sea_water_velocity ( time, zlev, lat, lon )
          units : m s-1
          __FillValue : -9999
          long_name : eastward component of velocity

       float northward_sea_water_velocity ( time, zlev, lat, lon )
          units : m s-1
          __FillValue : -9999
          long_name : northward component of velocity

       float sea_water_potential_temperature ( time, zlev, lat, lon )
          units : deg C
          __FillValue : -9999
          long_name : potential temperature

       float sea_water_salinity ( time, zlev, lat, lon )
          units : psu
          __FillValue : -9999
          long_name : salinity

I'm getting a lat/lon error similar to what I was seeing over the last
couple of days with the last "compliant" file data:
(0) check_for_y_lat_coord: Warning: Data either does not contain a valid
latitude coordinate array or d
oesn't contain one at all.
(0) A valid latitude coordinate array should have a 'units' attribute
equal to one of the following val
ues:
(0) 'degrees_north' 'degrees-north' 'degree_north' 'degrees north'
'degrees_N' 'Degrees_north' 'deg
ree_N' 'degreeN' 'degreesN' 'deg north'
(0) check_for_lon_coord: Warning: Data either does not contain a valid
longitude coordinate array or do
esn't contain one at all.
(0) A valid longitude coordinate array should have a 'units' attribute
equal to one of the following va
lues:
(0) 'degrees_east' 'degrees-east' 'degree_east' 'degrees east'
'degrees_E' 'Degrees_east' 'degree_E
' 'degreeE' 'degreesE' 'deg east'

I'm currently uploading MITGOM_STEP-3_FCAST-2.nc to the FTP site for
perusal, if y'all have time. I'm also uploading Step23-Model4.ncl.

Both are going into 'incoming'.

Thanks for all your support. I suspect this is another minor gotcha but
I'm lost.

gerry

On 10/06/2011 11:28 AM, Adam Phillips wrote:
> Hi Gerry,
> When you have 2D latitudes/longitudes, you need to assign the 2D lat/lon
> arrays as lat2d/lon2d attributes of your data array.
>
> I tried this with the data file you sent in:
>
> ncl 0> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"
> ncl 1> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"
> ncl 2> a = addfile("fcast_201009.nc","r")
> ncl 3> t = a->t(:,0,:,:)
> ncl 4> t@lat2d = a->alat
> ncl 5> t@lon2d = a->alon
> ncl 6> wks = gsn_open_wks("x11","test")
> ncl 7> plot = gsn_csm_contour_map(wks,t(0,:,:),False)
>
> and it worked. Let us know if you continue to have a problem plotting
> the data.. Adam
>
> On 10/06/2011 09:10 AM, Gerald Creager wrote:
>> I just tried but passive mode was refused (I've had this problem
>> recently here, too!). Donna or I will get a subset of one of the model
>> files uploaded, as well as the script shortly.
>>
>> Thanks, Gerry
>>
>>
>> On 10/06/2011 10:01 AM, Dennis Shea wrote:
>>> GC .. Send 2 or 3 sample files and a minimalistic script that isolates
>>> the issues you are having.
>>>
>>> ftp ftp.cgd.ucar.edu
>>> anonymous
>>> Use your email address for the password
>>> cd incoming
>>> put ...
>>> put ...
>>> :
>>> quit
>>>
>>> Send email to ncl-talk with the following information:
>>>
>>> The names of the files after you have successfully completed the transfer.
>>> We cannot look at the ftp/incoming directory for security reasons.
>>> The type of system you are on (uname -a).
>>>
>>> THX
>>> On 10/6/11 8:33 AM, Gerald Creager wrote:
>>>> I checked last night. We'*ARE* running v6.0.0:
>>>> [root@scoopdata gomexstep3]# ncl
>>>> Copyright (C) 1995-2011 - All Rights Reserved
>>>> University Corporation for Atmospheric Research
>>>> NCAR Command Language Version 6.0.0
>>>> The use of this software is governed by a License Agreement.
>>>> See http://www.ncl.ucar.edu/ for more details.
>>>>
>>>> but the problem persisted. Not a real problem. I'm looking for SST and
>>>> SSH at this point, so I snagged the top-level temperature and I'm good
>>>> on this.
>>>>
>>>> Now, however, another problem with the same netcdf input. The snippet of
>>>> code where I'm assigning variables for lat/lon:
>>>>
>>>> model2Lat=model2[:]->alat ; Degrees NORTH
>>>> model2Lon=model2[:]->alon ; Degrees WEST (?)
>>>>
>>>> And, a print statement to see what the vars look like:
>>>> print(model2Lat)
>>>>
>>>>
>>>> Variable: model2Lat
>>>> Type: float
>>>> Total Size: 3767796 bytes
>>>> 941949 values
>>>> Number of Dimensions: 2
>>>> Dimensions and sizes: [y | 2349] x [x | 401]
>>>> Coordinates:
>>>> Number Of Attributes: 3
>>>> long_name : latitude
>>>> units : deg-N
>>>> coordinates : alon alat
>>>> (0,0) 18
>>>> (0,1) 18
>>>> (0,2) 18
>>>> (0,3) 18
>>>> (0,4) 18
>>>> ...
>>>>
>>>> print(model2Lon)
>>>>
>>>>
>>>> Variable: model2Lon
>>>> Type: float
>>>> Total Size: 3767796 bytes
>>>> 941949 values
>>>> Number of Dimensions: 2
>>>> Dimensions and sizes: [y | 2349] x [x | 401]
>>>> Coordinates:
>>>> Number Of Attributes: 3
>>>> long_name : longitude
>>>> units : deg-W
>>>> coordinates : alon alat
>>>> (0,0) -98
>>>> (0,1) -97.95
>>>> (0,2) -97.9
>>>> (0,3) -97.85
>>>> (0,4) -97.8
>>>> (0,5) -97.75
>>>> (0,6) -97.7
>>>> ...
>>>>
>>>> OK, units are screwy:
>>>> model2Lat@units = "degrees_north"
>>>> model2Lon@units = "degrees_east"
>>>> print(model2Lat)
>>>>
>>>>
>>>> Variable: model2Lat
>>>> Type: float
>>>> Total Size: 3767796 bytes
>>>> 941949 values
>>>> Number of Dimensions: 2
>>>> Dimensions and sizes: [y | 2349] x [x | 401]
>>>> Coordinates:
>>>> Number Of Attributes: 3
>>>> long_name : latitude
>>>> units : degrees_north
>>>> coordinates : alon alat
>>>> (0,0) 18
>>>> (0,1) 18
>>>> (0,2) 18
>>>> (0,3) 18
>>>> ...
>>>> print(model2Lon)
>>>>
>>>>
>>>> Variable: model2Lon
>>>> Type: float
>>>> Total Size: 3767796 bytes
>>>> 941949 values
>>>> Number of Dimensions: 2
>>>> Dimensions and sizes: [y | 2349] x [x | 401]
>>>> Coordinates:
>>>> Number Of Attributes: 3
>>>> long_name : longitude
>>>> units : degrees_east
>>>> coordinates : alon alat
>>>> (0,0) -98
>>>> (0,1) -97.95
>>>> (0,2) -97.9
>>>> (0,3) -97.85
>>>> ...
>>>>
>>>> When I plot, I get a plot that isn't at the resolution I'm expecting,
>>>> and I get:
>>>> (0) check_for_y_lat_coord: Warning: Data either does not contain a valid
>>>> latitude coordinate array or doesn't co
>>>> ntain one at all.
>>>> (0) A valid latitude coordinate array should have a 'units' attribute
>>>> equal to one of the following values:
>>>> (0) 'degrees_north' 'degrees-north' 'degree_north' 'degrees north'
>>>> 'degrees_N' 'Degrees_north' 'degree_N' 'd
>>>> egreeN' 'degreesN' 'deg north'
>>>> (0) check_for_lon_coord: Warning: Data either does not contain a valid
>>>> longitude coordinate array or doesn't con
>>>> tain one at all.
>>>> (0) A valid longitude coordinate array should have a 'units' attribute
>>>> equal to one of the following values:
>>>> (0) 'degrees_east' 'degrees-east' 'degree_east' 'degrees east'
>>>> 'degrees_E' 'Degrees_east' 'degree_E' 'degree
>>>> E' 'degreesE' 'deg east'
>>>>
>>>> As near as I can tell, I've reassigned units in an appropriate manner.
>>>> I'm guessing this is is associate with gsn_csm_contour_map, but I've not
>>>> found where I've gone wrong.
>>>>
>>>> Thanks,
>>>> gerry
>>>>
>>>> On 10/05/2011 06:18 PM, Gerry Creager wrote:
>>>>> I'll give it a try in a little while (parental duties right now!)
>>>>>
>>>>> gc
>>>>>
>>>>> Adam Phillips wrote:
>>>>>> Hi Gerry,
>>>>>> The newest version of ncl (v6.0.0) allows array sizes larger than 2GB.
>>>>>> Previous versions do not. I am guessing you are using a pre-6.0.0
>>>>>> version and your aggregated T array is (potentially) larger than 2GB..
>>>>>>
>>>>>> If I'm off on my guess let us know.. Adam
>>>>>>
>>>>>> On 10/05/2011 04:50 PM, Gerald Creager wrote:
>>>>>>> Thanks, Gentlemen. I didn't realize that (but I should have)...
>>>>>>>
>>>>>>> OK, so I tried this:
>>>>>>> model2Sst=model2[:]->t ; potential temperature with
>>>>>>> coordinates : time, zz, y, x
>>>>>>> fatal:Aggregating variable t from file list variable model2 as
>>>>>>> specified
>>>>>>> would exceed maximum NCL variable size
>>>>>>>
>>>>>>> gerry
>>>>>>>
>>>>>>> On 10/05/2011 04:45 PM, David Brown wrote:
>>>>>>>> Hi Gerry,
>>>>>>>> When you use 'addfiles' rather than 'addfile', you get a file list
>>>>>>>> variable rather than a file variable. It requires a different syntax.
>>>>>>>> Try
>>>>>>>> model2Sst=model2[:]->t
>>>>>>>> and
>>>>>>>> Time = model2[:]->time
>>>>>>>>
>>>>>>>> You can say 'model2[0]' in order to "reference a file".
>>>>>>>> -dave
>>>>>>>>
>>>>>>>> On Oct 5, 2011, at 3:09 PM, Gerald Creager wrote:
>>>>>>>>
>>>>>>>>> I'm trying to assign values to a variable from a netcdf file
>>>>>>>>> (ncl_filedump output for the file is attached). A sample snippet
>>>>>>>>> of code is attached. Note the error... And, note that line 6
>>>>>>>>> does, in fact load the data into 'model2'.
>>>>>>>>>
>>>>>>>>> ncl 30> model2Sst=model2->t
>>>>>>>>> fatal:(model2) does not reference a filencl 0> load
>>>>>>>>> "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"
>>>>>>>>> ncl 1> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"
>>>>>>>>> ncl 2> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"
>>>>>>>>> ncl 3> load
>>>>>>>>> "$NCARG_ROOT/lib/ncarg/nclscripts/contrib/calendar_decode2.ncl"
>>>>>>>>> ncl 4>
>>>>>>>>> ncl 5> fnameModel2=systemfunc ("ls fcast_20100*.nc") ; files
>>>>>>>>> location
>>>>>>>>> ncl 6> model2=addfiles(fnameModel2, "r")
>>>>>>>>> ncl 7> type="x11"
>>>>>>>>> ncl 8> Time=model2->time
>>>>>>>>> fatal:(model2) does not reference a file
>>>>>>>>> fatal:Execute: Error occurred at or near line 8
>>>>>>>>>
>>>>>>>>> fatal:Execute: Error occurred at or near line 30
>>>>>>>>>
>>>>>>>>> Thanks, Gerry
>>>>>>>>> -
>>>>>>>>> Gerry Creager -- gerry.creager@tamu.edu
>>>>>>>>> Texas Mesonet -- AATLT, Texas A&M University
>>>>>>>>> Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.862.3983
>>>>>>>>> Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX
>>>>>>>>> 77843
>>>>>>>>> <nclfiledump-model2.txt>_______________________________________________
>>>>>>>>>
>>>>>>>>> ncl-talk mailing list
>>>>>>>>> List instructions, subscriber options, unsubscribe:
>>>>>>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>>>
>>>>>
>>>>
>>
>

-- 
Gerry Creager -- gerry.creager@tamu.edu
Texas Mesonet -- AATLT, Texas A&M University
Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.862.3983
Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX 77843
_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Fri Oct 7 14:00:00 2011

This archive was generated by hypermail 2.1.8 : Sun Oct 09 2011 - 13:05:26 MDT