Re: Variable input error: what am I doing wrong?

From: David Brown <dbrown_at_nyahnyahspammersnyahnyah>
Date: Fri Oct 07 2011 - 17:03:01 MDT

You need to set cnLevelSelectionMode to "ManualLevels" for the cnMin/MaxLevelValF resources to have any effect. Also you should set cnLevelSpacingF.
For the script as you sent it to me, the maximum value is returned as 65.1459, so it appears in that instance the _FillValue you set did take effect.
However, in the text below it looks like there are two underscores in the attribute name '__FillValue', rather than '_FillValue'. That won't work.
The double underscore version shows up in the list of attributes below as well, although there it is with a value of -9999.
Hope this helps.
 -dave

On Oct 7, 2011, at 4:29 PM, Gerald Creager wrote:

> And another question: Despite claiming a fill-value for SSH is -9999,
> they've actually populated either fill, or missing, with 999900. I've
> added:
> model4Ssh@__FillValue = 999900
>
> but it doesn't seem to do much good. I've also tried to limit the range
> of my output thus:
> res@cnMinLevelValF = -100 ; Minimum SSH variation from MSL
> res@cnMaxLevelValF = 80 ; Maximum SSH above MSL
>
> For reference...
> ncl 92> print(min(model4Ssh))
> (0) -57.33852
> ncl 93> print(max(model4Ssh))
> (0) 999900
>
> Thanks!
> gerry
>
>
> On 10/07/2011 04:47 PM, David Brown wrote:
>> Hi Gerry,
>>
>> Your problem stems from the fact that although the file claims to be CF-1.0 compliant, it is not quite. The dimensions are named "lat" and "lon" but the intended coordinate variables are named "latitude" and "longitude". The CF convention is that a variable is a (1D) coordinate variable only if it has the same name as the dimension. Doing a printVarSummary on the data variable you get this:
>>
>> Variable: model4Ssh
>> Type: float
>> Total Size: 41445756 bytes
>> 10361439 values
>> Number of Dimensions: 3
>> Dimensions and sizes: [time | 99] x [lat | 261] x [lon | 401]
>> Coordinates:
>> time: [734776.5.. 0]
>> Number Of Attributes: 5
>> _FillValue : 999900
>> units : m
>> __FillValue : -9999
>> long_name : sea-surface height
>> positive : up
>>
>> Notice the dimensions are named but no coordinates are associated with lat and lon. The fix is simply to add these two lines after you read in model4SSh
>>
>> model4Ssh&lat = model4Lat
>> model4Ssh&lon = model4Lon
>>
>> -dave
>>
>> On Oct 7, 2011, at 1:59 PM, Gerald Creager wrote:
>>
>>> Adam, I'm back....
>>>
>>> We're working on a project with all sorts of purported NetCDF files I'm
>>> getting to sort through. Just as an idea, the one I'm on now uses Matlab
>>> datenum for its time variable format; that's a solved problem.
>>>
>>> It does have 1d lat/lon data as evidenced by the data from ncl_filedump
>>> shown below:
>>> [gerry@wagoneer Step3]$ ncl_filedump MITGOM_STEP-3_FCAST-2.nc
>>> Copyright (C) 1995-2011 - All Rights Reserved
>>> University Corporation for Atmospheric Research
>>> NCAR Command Language Version 6.0.0
>>> The use of this software is governed by a License Agreement.
>>> See http://www.ncl.ucar.edu/ for more details.
>>>
>>> Variable: f
>>> Type: file
>>> filename: MITGOM_STEP-3_FCAST-2
>>> path: MITGOM_STEP-3_FCAST-2.nc
>>> file global attributes:
>>> title : MITgcm GoM three-month forecast for GOMEX-PPP STEP-3:2011
>>> Model : MITgcm
>>> Filename :
>>> /net/wave3/ganeshgopal/WAVE5_DATA/GOM_4DVAR_2011_RUNS/run_23jul2011/res_nc/MITGOM_STEP-3_FCAST-2.nc
>>> Model_Resolution : 1/10 degree
>>> Hgrid : Arakawa-C Grid
>>> Stdgrid : Interpolated to standard grid : [262E-282E, 18N-31N] :
>>> 0.05 deg resolution : ( IM=401, JM=201 )
>>> Vgrid : Z - level (40 levels)
>>> Stdlevel : Interpolated to standard level: 22 levels ( 0, -20,
>>> -40, -60, -80, -100, -150, -200, -250, -300, -400, -500, -600, -700,
>>> -800, -900, -1000, -1500, -2000, -2500, -3000, -3500 m )
>>> Institution : Scripps Institution of Oceanography
>>> Conventions : CF-1.0
>>> Contact : Bruce Cornuelle (bdc@ucsd.edu), Ganesh Gopal
>>> (ggopalak@ucsd.edu)
>>> dimensions:
>>> lon = 401
>>> lat = 261
>>> zlev = 22
>>> time = 99 // unlimited
>>> variables:
>>> float longitude ( lon )
>>> units : degrees_east
>>> long_name : longitude at cell-center
>>>
>>> float latitude ( lat )
>>> units : degrees_north
>>> long_name : latitude at cell-center
>>>
>>> float zlev ( zlev )
>>> units : m
>>> long_name : depth at cell-center
>>> negative : down
>>>
>>> float time ( time )
>>> units : matlab datenum
>>> long_name : date
>>>
>>> float sea_floor_depth_below_sea_level ( lat, lon )
>>> units : m
>>> __FillValue : -9999
>>> long_name : depth
>>> positive : down
>>>
>>> float sea_surface_height_above_sea_level ( time, lat, lon )
>>> units : m
>>> __FillValue : -9999
>>> long_name : sea-surface height
>>> positive : up
>>>
>>> float eastward_sea_water_velocity_at_bottom ( time, lat, lon )
>>> units : m s-1
>>> __FillValue : -9999
>>> long_name : bottom u
>>>
>>> float northward_sea_water_velocity_at_bottom ( time, lat, lon )
>>> units : m s-1
>>> __FillValue : -9999
>>> long_name : bottom v
>>>
>>> float temp_at_bottom ( time, lat, lon )
>>> units : deg C
>>> __FillValue : -9999
>>> long_name : bottom temp
>>>
>>> float salinity_at_bottom ( time, lat, lon )
>>> units : psu
>>> __FillValue : -9999
>>> long_name : bottom salinity
>>>
>>> float eastward_sea_water_velocity ( time, zlev, lat, lon )
>>> units : m s-1
>>> __FillValue : -9999
>>> long_name : eastward component of velocity
>>>
>>> float northward_sea_water_velocity ( time, zlev, lat, lon )
>>> units : m s-1
>>> __FillValue : -9999
>>> long_name : northward component of velocity
>>>
>>> float sea_water_potential_temperature ( time, zlev, lat, lon )
>>> units : deg C
>>> __FillValue : -9999
>>> long_name : potential temperature
>>>
>>> float sea_water_salinity ( time, zlev, lat, lon )
>>> units : psu
>>> __FillValue : -9999
>>> long_name : salinity
>>>
>>>
>>> I'm getting a lat/lon error similar to what I was seeing over the last
>>> couple of days with the last "compliant" file data:
>>> (0) check_for_y_lat_coord: Warning: Data either does not contain a valid
>>> latitude coordinate array or d
>>> oesn't contain one at all.
>>> (0) A valid latitude coordinate array should have a 'units' attribute
>>> equal to one of the following val
>>> ues:
>>> (0) 'degrees_north' 'degrees-north' 'degree_north' 'degrees north'
>>> 'degrees_N' 'Degrees_north' 'deg
>>> ree_N' 'degreeN' 'degreesN' 'deg north'
>>> (0) check_for_lon_coord: Warning: Data either does not contain a valid
>>> longitude coordinate array or do
>>> esn't contain one at all.
>>> (0) A valid longitude coordinate array should have a 'units' attribute
>>> equal to one of the following va
>>> lues:
>>> (0) 'degrees_east' 'degrees-east' 'degree_east' 'degrees east'
>>> 'degrees_E' 'Degrees_east' 'degree_E
>>> ' 'degreeE' 'degreesE' 'deg east'
>>>
>>>
>>> I'm currently uploading MITGOM_STEP-3_FCAST-2.nc to the FTP site for
>>> perusal, if y'all have time. I'm also uploading Step23-Model4.ncl.
>>>
>>> Both are going into 'incoming'.
>>>
>>> Thanks for all your support. I suspect this is another minor gotcha but
>>> I'm lost.
>>>
>>> gerry
>>>
>>> On 10/06/2011 11:28 AM, Adam Phillips wrote:
>>>> Hi Gerry,
>>>> When you have 2D latitudes/longitudes, you need to assign the 2D lat/lon
>>>> arrays as lat2d/lon2d attributes of your data array.
>>>>
>>>> I tried this with the data file you sent in:
>>>>
>>>> ncl 0> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"
>>>> ncl 1> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"
>>>> ncl 2> a = addfile("fcast_201009.nc","r")
>>>> ncl 3> t = a->t(:,0,:,:)
>>>> ncl 4> t@lat2d = a->alat
>>>> ncl 5> t@lon2d = a->alon
>>>> ncl 6> wks = gsn_open_wks("x11","test")
>>>> ncl 7> plot = gsn_csm_contour_map(wks,t(0,:,:),False)
>>>>
>>>> and it worked. Let us know if you continue to have a problem plotting
>>>> the data.. Adam
>>>>
>>>> On 10/06/2011 09:10 AM, Gerald Creager wrote:
>>>>> I just tried but passive mode was refused (I've had this problem
>>>>> recently here, too!). Donna or I will get a subset of one of the model
>>>>> files uploaded, as well as the script shortly.
>>>>>
>>>>> Thanks, Gerry
>>>>>
>>>>>
>>>>> On 10/06/2011 10:01 AM, Dennis Shea wrote:
>>>>>> GC .. Send 2 or 3 sample files and a minimalistic script that isolates
>>>>>> the issues you are having.
>>>>>>
>>>>>> ftp ftp.cgd.ucar.edu
>>>>>> anonymous
>>>>>> Use your email address for the password
>>>>>> cd incoming
>>>>>> put ...
>>>>>> put ...
>>>>>> :
>>>>>> quit
>>>>>>
>>>>>> Send email to ncl-talk with the following information:
>>>>>>
>>>>>> The names of the files after you have successfully completed the transfer.
>>>>>> We cannot look at the ftp/incoming directory for security reasons.
>>>>>> The type of system you are on (uname -a).
>>>>>>
>>>>>> THX
>>>>>> On 10/6/11 8:33 AM, Gerald Creager wrote:
>>>>>>> I checked last night. We'*ARE* running v6.0.0:
>>>>>>> [root@scoopdata gomexstep3]# ncl
>>>>>>> Copyright (C) 1995-2011 - All Rights Reserved
>>>>>>> University Corporation for Atmospheric Research
>>>>>>> NCAR Command Language Version 6.0.0
>>>>>>> The use of this software is governed by a License Agreement.
>>>>>>> See http://www.ncl.ucar.edu/ for more details.
>>>>>>>
>>>>>>> but the problem persisted. Not a real problem. I'm looking for SST and
>>>>>>> SSH at this point, so I snagged the top-level temperature and I'm good
>>>>>>> on this.
>>>>>>>
>>>>>>> Now, however, another problem with the same netcdf input. The snippet of
>>>>>>> code where I'm assigning variables for lat/lon:
>>>>>>>
>>>>>>> model2Lat=model2[:]->alat ; Degrees NORTH
>>>>>>> model2Lon=model2[:]->alon ; Degrees WEST (?)
>>>>>>>
>>>>>>> And, a print statement to see what the vars look like:
>>>>>>> print(model2Lat)
>>>>>>>
>>>>>>>
>>>>>>> Variable: model2Lat
>>>>>>> Type: float
>>>>>>> Total Size: 3767796 bytes
>>>>>>> 941949 values
>>>>>>> Number of Dimensions: 2
>>>>>>> Dimensions and sizes: [y | 2349] x [x | 401]
>>>>>>> Coordinates:
>>>>>>> Number Of Attributes: 3
>>>>>>> long_name : latitude
>>>>>>> units : deg-N
>>>>>>> coordinates : alon alat
>>>>>>> (0,0) 18
>>>>>>> (0,1) 18
>>>>>>> (0,2) 18
>>>>>>> (0,3) 18
>>>>>>> (0,4) 18
>>>>>>> ...
>>>>>>>
>>>>>>> print(model2Lon)
>>>>>>>
>>>>>>>
>>>>>>> Variable: model2Lon
>>>>>>> Type: float
>>>>>>> Total Size: 3767796 bytes
>>>>>>> 941949 values
>>>>>>> Number of Dimensions: 2
>>>>>>> Dimensions and sizes: [y | 2349] x [x | 401]
>>>>>>> Coordinates:
>>>>>>> Number Of Attributes: 3
>>>>>>> long_name : longitude
>>>>>>> units : deg-W
>>>>>>> coordinates : alon alat
>>>>>>> (0,0) -98
>>>>>>> (0,1) -97.95
>>>>>>> (0,2) -97.9
>>>>>>> (0,3) -97.85
>>>>>>> (0,4) -97.8
>>>>>>> (0,5) -97.75
>>>>>>> (0,6) -97.7
>>>>>>> ...
>>>>>>>
>>>>>>> OK, units are screwy:
>>>>>>> model2Lat@units = "degrees_north"
>>>>>>> model2Lon@units = "degrees_east"
>>>>>>> print(model2Lat)
>>>>>>>
>>>>>>>
>>>>>>> Variable: model2Lat
>>>>>>> Type: float
>>>>>>> Total Size: 3767796 bytes
>>>>>>> 941949 values
>>>>>>> Number of Dimensions: 2
>>>>>>> Dimensions and sizes: [y | 2349] x [x | 401]
>>>>>>> Coordinates:
>>>>>>> Number Of Attributes: 3
>>>>>>> long_name : latitude
>>>>>>> units : degrees_north
>>>>>>> coordinates : alon alat
>>>>>>> (0,0) 18
>>>>>>> (0,1) 18
>>>>>>> (0,2) 18
>>>>>>> (0,3) 18
>>>>>>> ...
>>>>>>> print(model2Lon)
>>>>>>>
>>>>>>>
>>>>>>> Variable: model2Lon
>>>>>>> Type: float
>>>>>>> Total Size: 3767796 bytes
>>>>>>> 941949 values
>>>>>>> Number of Dimensions: 2
>>>>>>> Dimensions and sizes: [y | 2349] x [x | 401]
>>>>>>> Coordinates:
>>>>>>> Number Of Attributes: 3
>>>>>>> long_name : longitude
>>>>>>> units : degrees_east
>>>>>>> coordinates : alon alat
>>>>>>> (0,0) -98
>>>>>>> (0,1) -97.95
>>>>>>> (0,2) -97.9
>>>>>>> (0,3) -97.85
>>>>>>> ...
>>>>>>>
>>>>>>> When I plot, I get a plot that isn't at the resolution I'm expecting,
>>>>>>> and I get:
>>>>>>> (0) check_for_y_lat_coord: Warning: Data either does not contain a valid
>>>>>>> latitude coordinate array or doesn't co
>>>>>>> ntain one at all.
>>>>>>> (0) A valid latitude coordinate array should have a 'units' attribute
>>>>>>> equal to one of the following values:
>>>>>>> (0) 'degrees_north' 'degrees-north' 'degree_north' 'degrees north'
>>>>>>> 'degrees_N' 'Degrees_north' 'degree_N' 'd
>>>>>>> egreeN' 'degreesN' 'deg north'
>>>>>>> (0) check_for_lon_coord: Warning: Data either does not contain a valid
>>>>>>> longitude coordinate array or doesn't con
>>>>>>> tain one at all.
>>>>>>> (0) A valid longitude coordinate array should have a 'units' attribute
>>>>>>> equal to one of the following values:
>>>>>>> (0) 'degrees_east' 'degrees-east' 'degree_east' 'degrees east'
>>>>>>> 'degrees_E' 'Degrees_east' 'degree_E' 'degree
>>>>>>> E' 'degreesE' 'deg east'
>>>>>>>
>>>>>>> As near as I can tell, I've reassigned units in an appropriate manner.
>>>>>>> I'm guessing this is is associate with gsn_csm_contour_map, but I've not
>>>>>>> found where I've gone wrong.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> gerry
>>>>>>>
>>>>>>> On 10/05/2011 06:18 PM, Gerry Creager wrote:
>>>>>>>> I'll give it a try in a little while (parental duties right now!)
>>>>>>>>
>>>>>>>> gc
>>>>>>>>
>>>>>>>> Adam Phillips wrote:
>>>>>>>>> Hi Gerry,
>>>>>>>>> The newest version of ncl (v6.0.0) allows array sizes larger than 2GB.
>>>>>>>>> Previous versions do not. I am guessing you are using a pre-6.0.0
>>>>>>>>> version and your aggregated T array is (potentially) larger than 2GB..
>>>>>>>>>
>>>>>>>>> If I'm off on my guess let us know.. Adam
>>>>>>>>>
>>>>>>>>> On 10/05/2011 04:50 PM, Gerald Creager wrote:
>>>>>>>>>> Thanks, Gentlemen. I didn't realize that (but I should have)...
>>>>>>>>>>
>>>>>>>>>> OK, so I tried this:
>>>>>>>>>> model2Sst=model2[:]->t ; potential temperature with
>>>>>>>>>> coordinates : time, zz, y, x
>>>>>>>>>> fatal:Aggregating variable t from file list variable model2 as
>>>>>>>>>> specified
>>>>>>>>>> would exceed maximum NCL variable size
>>>>>>>>>>
>>>>>>>>>> gerry
>>>>>>>>>>
>>>>>>>>>> On 10/05/2011 04:45 PM, David Brown wrote:
>>>>>>>>>>> Hi Gerry,
>>>>>>>>>>> When you use 'addfiles' rather than 'addfile', you get a file list
>>>>>>>>>>> variable rather than a file variable. It requires a different syntax.
>>>>>>>>>>> Try
>>>>>>>>>>> model2Sst=model2[:]->t
>>>>>>>>>>> and
>>>>>>>>>>> Time = model2[:]->time
>>>>>>>>>>>
>>>>>>>>>>> You can say 'model2[0]' in order to "reference a file".
>>>>>>>>>>> -dave
>>>>>>>>>>>
>>>>>>>>>>> On Oct 5, 2011, at 3:09 PM, Gerald Creager wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I'm trying to assign values to a variable from a netcdf file
>>>>>>>>>>>> (ncl_filedump output for the file is attached). A sample snippet
>>>>>>>>>>>> of code is attached. Note the error... And, note that line 6
>>>>>>>>>>>> does, in fact load the data into 'model2'.
>>>>>>>>>>>>
>>>>>>>>>>>> ncl 30> model2Sst=model2->t
>>>>>>>>>>>> fatal:(model2) does not reference a filencl 0> load
>>>>>>>>>>>> "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"
>>>>>>>>>>>> ncl 1> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"
>>>>>>>>>>>> ncl 2> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"
>>>>>>>>>>>> ncl 3> load
>>>>>>>>>>>> "$NCARG_ROOT/lib/ncarg/nclscripts/contrib/calendar_decode2.ncl"
>>>>>>>>>>>> ncl 4>
>>>>>>>>>>>> ncl 5> fnameModel2=systemfunc ("ls fcast_20100*.nc") ; files
>>>>>>>>>>>> location
>>>>>>>>>>>> ncl 6> model2=addfiles(fnameModel2, "r")
>>>>>>>>>>>> ncl 7> type="x11"
>>>>>>>>>>>> ncl 8> Time=model2->time
>>>>>>>>>>>> fatal:(model2) does not reference a file
>>>>>>>>>>>> fatal:Execute: Error occurred at or near line 8
>>>>>>>>>>>>
>>>>>>>>>>>> fatal:Execute: Error occurred at or near line 30
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks, Gerry
>>>>>>>>>>>> -
>>>>>>>>>>>> Gerry Creager -- gerry.creager@tamu.edu
>>>>>>>>>>>> Texas Mesonet -- AATLT, Texas A&M University
>>>>>>>>>>>> Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.862.3983
>>>>>>>>>>>> Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX
>>>>>>>>>>>> 77843
>>>>>>>>>>>> <nclfiledump-model2.txt>_______________________________________________
>>>>>>>>>>>>
>>>>>>>>>>>> ncl-talk mailing list
>>>>>>>>>>>> List instructions, subscriber options, unsubscribe:
>>>>>>>>>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>
>>>
>>> --
>>> Gerry Creager -- gerry.creager@tamu.edu
>>> Texas Mesonet -- AATLT, Texas A&M University
>>> Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.862.3983
>>> Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX 77843
>>> _______________________________________________
>>> ncl-talk mailing list
>>> List instructions, subscriber options, unsubscribe:
>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>
>
> --
> Gerry Creager -- gerry.creager@tamu.edu
> Texas Mesonet -- AATLT, Texas A&M University
> Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.862.3983
> Office: 7607 Eastmark Drive, Suite 112, College Station, TX 77840
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Fri Oct 7 17:03:09 2011

This archive was generated by hypermail 2.1.8 : Sun Oct 09 2011 - 13:05:26 MDT