That's great David -- thanks for the reply. Now I understand why I was getting an error.
You guys do an amazing job with this maillist.
Much appreciated.
Sam
On 04/03/2013 10:37 AM, David Brown wrote:
> Hi Sam,
> I apologize that we did not answer this question in a more timely manner. The fact is that you do subsetting using normal NCL syntax. When you do
>
> ncl 0> f = addfile("http://coastwatch.pfeg.noaa.gov/erddap/griddap/hawaii_9d34_0512_2367","r")
>
> NCL only reads the metadata and the coordinate values. It does not read any actual data.
>
> ncl 1> print(f)
>
> Variable: f
> Type: file
> filename: hawaii_9d34_0512_2367
> path: http://coastwatch.pfeg.noaa.gov/erddap/griddap/hawaii_9d34_0512_2367
> file global attributes:
> cdm_data_type : Grid
> Conventions : COARDS, CF-1.6, Unidata Dataset Discovery v1.0
> dataType : Grid
> documentation : http://apdrc.soest.hawaii.edu/datadoc/soda_2.1.6.php
> Easternmost_Easting : 359.75
> geospatial_lat_max : 89.25
> geospatial_lat_min : -75.25
> geospatial_lat_resolution : 0.5
> geospatial_lat_units : degrees_north
> geospatial_lon_max : 359.75
> geospatial_lon_min : 0.25
> geospatial_lon_resolution : 0.5
> ….
>
> You can read a subset of a variable just as you normally would using NCL syntax for file access:
>
> ncl 2> v = f->v(0,0,{-20:20},{0:360})
>
> ncl 3> printVarSummary(v)
>
> Variable: v
> Type: float
> Total Size: 230400 bytes
> 57600 values
> Number of Dimensions: 2
> Dimensions and sizes: [latitude | 80] x [longitude | 720]
> Coordinates:
> latitude: [-19.75..19.75]
> longitude: [0.25..359.75]
> Number Of Attributes: 10
> altitude : -5
> time : -377481600
> _FillValue : -9.99e+33
> colorBarMaximum : 0.5
> colorBarMinimum : -0.5
> ioos_category : Currents
> long_name : Northward Sea Water Velocity
> missing_value : -9.99e+33
> standard_name : northward_sea_water_velocity
> units : m s-1
>
> Try reading the whole variable 'v' in this dataset if you want proof that NCL does not read data until it is explicitly asked for. Because the full variable is bigger than OPeNDAP can handle (2GB), it will give an error.
>
> Hope this helps, and again I apologize for the delay in answering.
> -dave
>
>
>
> On Mar 11, 2013, at 6:20 PM, Sam McClatchie (NOAA Federal) <sam.mcclatchie@noaa.gov> wrote:
>
>> Colleagues
>>
>> I am having problems sub-setting the SODA model output using opendap in NCL and I am wondering if I am using the wrong sub-setting protocol.
>>
>> A dds query of the data set gives:
>>
>> Dataset {
>> Float64 time[time = 612];
>> Float64 altitude[altitude = 40];
>> Float64 latitude[latitude = 330];
>> Float64 longitude[longitude = 720];
>> GRID {
>> ARRAY:
>> Float32 temp[time = 612][altitude = 40][latitude = 330][longitude = 720];
>> MAPS:
>> Float64 time[time = 612];
>> Float64 altitude[altitude = 40];
>> Float64 latitude[latitude = 330];
>> Float64 longitude[longitude = 720];
>> } temp;
>> GRID {
>> ARRAY:
>> Float32 salt[time = 612][altitude = 40][latitude = 330][longitude = 720];
>> MAPS:
>> Float64 time[time = 612];
>> Float64 altitude[altitude = 40];
>> Float64 latitude[latitude = 330];
>> Float64 longitude[longitude = 720];
>> } salt;
>> GRID {
>> ARRAY:
>> Float32 u[time = 612][altitude = 40][latitude = 330][longitude = 720];
>> MAPS:
>> Float64 time[time = 612];
>> Float64 altitude[altitude = 40];
>> Float64 latitude[latitude = 330];
>> Float64 longitude[longitude = 720];
>> } u;
>> GRID {
>> ARRAY:
>> Float32 v[time = 612][altitude = 40][latitude = 330][longitude = 720];
>> MAPS:
>> Float64 time[time = 612];
>> Float64 altitude[altitude = 40];
>> Float64 latitude[latitude = 330];
>> Float64 longitude[longitude = 720];
>> } v;
>> } hawaii_9d34_0512_2367;
>>
>> A DAS query of the data set gives
>> Attributes {
>> time {
>> String _CoordinateAxisType "Time";
>> Float64 actual_range -3.774816e+8, 1.2292992e+9;
>> String axis "T";
>> String ioos_category "Time";
>> String long_name "Centered Time";
>> String standard_name "time";
>> String time_origin "01-JAN-1970 00:00:00";
>> String units "seconds since 1970-01-01T00:00:00Z";
>> }
>> altitude {
>> String _CoordinateAxisType "Height";
>> String _CoordinateZisPositive "up";
>> Float64 actual_range -5.0, -5374.0;
>> String axis "Z";
>> String ioos_category "Location";
>> String long_name "Altitude";
>> String positive "up";
>> String standard_name "altitude";
>> String units "m";
>> }
>> latitude {
>> String _CoordinateAxisType "Lat";
>> Float64 actual_range -75.25, 89.25;
>> String axis "Y";
>> String ioos_category "Location";
>> String long_name "Latitude";
>> String standard_name "latitude";
>> String units "degrees_north";
>> }
>> longitude {
>> String _CoordinateAxisType "Lon";
>> Float64 actual_range 0.25, 359.75;
>> String axis "X";
>> String ioos_category "Location";
>> String long_name "Longitude";
>> String standard_name "longitude";
>> String units "degrees_east";
>> }
>> temp {
>> Float32 _FillValue -9.99E33;
>> Float64 colorBarMaximum 32.0;
>> Float64 colorBarMinimum 0.0;
>> String ioos_category "Temperature";
>> String long_name "Sea Water Temperature";
>> Float32 missing_value -9.99E33;
>> String standard_name "sea_water_temperature";
>> String units "degree_C";
>> }
>> salt {
>> Float32 _FillValue -9.99E33;
>> Float64 colorBarMaximum 37.0;
>> Float64 colorBarMinimum 32.0;
>> String ioos_category "Salinity";
>> String long_name "Salinity";
>> Float32 missing_value -9.99E33;
>> String standard_name "sea_water_salinity";
>> String units "PSU";
>> }
>> u {
>> Float32 _FillValue -9.99E33;
>> Float64 colorBarMaximum 0.5;
>> Float64 colorBarMinimum -0.5;
>> String ioos_category "Currents";
>> String long_name "Eastward Sea Water Velocity";
>> Float32 missing_value -9.99E33;
>> String standard_name "eastward_sea_water_velocity";
>> String units "m s-1";
>> }
>> v {
>> Float32 _FillValue -9.99E33;
>> Float64 colorBarMaximum 0.5;
>> Float64 colorBarMinimum -0.5;
>> String ioos_category "Currents";
>> String long_name "Northward Sea Water Velocity";
>> Float32 missing_value -9.99E33;
>> String standard_name "northward_sea_water_velocity";
>> String units "m s-1";
>> }
>> NC_GLOBAL {
>> String cdm_data_type "Grid";
>> String Conventions "COARDS, CF-1.6, Unidata Dataset Discovery v1.0";
>> String dataType "Grid";
>> String documentation "http://apdrc.soest.hawaii.edu/datadoc/soda_2.1.6.php";
>> Float64 Easternmost_Easting 359.75;
>> Float64 geospatial_lat_max 89.25;
>> Float64 geospatial_lat_min -75.25;
>> Float64 geospatial_lat_resolution 0.5;
>> String geospatial_lat_units "degrees_north";
>> Float64 geospatial_lon_max 359.75;
>> Float64 geospatial_lon_min 0.25;
>> Float64 geospatial_lon_resolution 0.5;
>> String geospatial_lon_units "degrees_east";
>> Float64 geospatial_vertical_max -5.0;
>> Float64 geospatial_vertical_min -5374.0;
>> String geospatial_vertical_positive "up";
>> String geospatial_vertical_units "m";
>> String history "Tue Feb 26 12:55:18 HST 2013 : imported by GrADS Data Server 2.0
>> 2013-03-11T23:17:53Z http://apdrc.soest.hawaii.edu/dods/public_data/SODA/soda_pop2.1.6
>> 2013-03-11T23:17:53Z http://coastwatch.pfeg.noaa.gov/erddap/griddap/hawaii_9d34_0512_2367.das";
>> String infoUrl "http://www.atmos.umd.edu/~ocean/";
>> String institution "TAMU/UMD";
>> String keywords "Oceans > Ocean Circulation > Ocean Currents,
>> Oceans > Ocean Temperature > Water Temperature,
>> Oceans > Salinity/Density > Salinity,
>> circulation, currents, density, depths, eastward, eastward_sea_water_velocity, means, monthly, northward, northward_sea_water_velocity, ocean, oceans, pop, salinity, sea, sea_water_salinity, sea_water_temperature, seawater, soda, tamu, temperature, umd, velocity, water";
>> String keywords_vocabulary "GCMD Science Keywords";
>> String license "The data may be used and redistributed for free but is not intended
>> for legal use, since it may contain inaccuracies. Neither the data
>> Contributor, ERD, NOAA, nor the United States Government, nor any
>> of their employees or contractors, makes any warranty, express or
>> implied, including warranties of merchantability and fitness for a
>> particular purpose, or assumes any legal liability for the accuracy,
>> completeness, or usefulness, of this information.";
>> String Metadata_Conventions "COARDS, CF-1.6, Unidata Dataset Discovery v1.0";
>> Float64 Northernmost_Northing 89.25;
>> String sourceUrl "http://apdrc.soest.hawaii.edu/dods/public_data/SODA/soda_pop2.1.6";
>> Float64 Southernmost_Northing -75.25;
>> String standard_name_vocabulary "CF-12";
>> String summary "Simple Ocean Data Assimilation (SODA) version 2.1.6 - A reanalysis of ocean
>> climate. SODA uses the GFDL modular ocean model version 2.2. The model is
>> forced by observed surface wind stresses from the COADS data set (from 1958
>> to 1992) and from NCEP (after 1992). Note that the wind stresses were
>> detrended before use due to inconsistencies with observed sea level pressure
>> trends. The model is also constrained by constant assimilation of observed
>> temperatures, salinities, and altimetry using an optimal data assimilation
>> technique. The observed data comes from: 1) The World Ocean Atlas 1994 which
>> contains ocean temperatures and salinities from mechanical
>> bathythermographs, expendable bathythermographs and conductivity-temperature-
>> depth probes. 2) The expendable bathythermograph archive 3) The TOGA-TAO
>> thermistor array 4) The Soviet SECTIONS tropical program 5) Satellite
>> altimetry from Geosat, ERS/1 and TOPEX/Poseidon.
>> We are now exploring an eddy-permitting reanalysis based on the Parallel
>> Ocean Program POP-1.4 model with 40 levels in the vertical and a 0.4x0.25
>> degree displaced pole grid (25 km resolution in the western North
>> Atlantic). The first version of this we will release is SODA1.2, a
>> reanalysis driven by ERA-40 winds covering the period 1958-2001 (extended to
>> the current year using available altimetry).";
>> String time_coverage_end "2008-12-15T00:00:00Z";
>> String time_coverage_start "1958-01-15T00:00:00Z";
>> String title "SODA - POP 2.1.6 Monthly Means (At Depths)";
>> Float64 Westernmost_Easting 0.25;
>> }
>> }
>> Accessing the entire dataset using NCL works fine:
>> ;; download SODA model with openDAP
>> url="http://coastwatch.pfeg.noaa.gov/erddap/griddap/hawaii_9d34_0512_2367"
>> filename = url
>> exists = isfilepresent(filename)
>> if(.not.exists) then
>> print("OPeNDAP isfilepresent test unsuccessful.")
>> print("Either file doesn't exist, or NCL does not have OPeNDAP capabilities on this system")
>> else
>> print("OPeNDAP isfilepresent test successful.")
>> f = addfile(filename,"r")
>> vars = getfilevarnames(f)
>> print(vars)
>> end if
>>
>> which gives me:
>> (0) OPeNDAP isfilepresent test successful.
>>
>> Variable: vars
>> Type: string
>> Total Size: 64 bytes
>> 8 values
>> Number of Dimensions: 1
>> Dimensions and sizes: [8]
>> Coordinates:
>> (0) time
>> (1) altitude
>> (2) latitude
>> (3) longitude
>> (4) temp
>> (5) salt
>> (6) u
>> (7) v
>> ncl 46>
>>
>> But when I add the sub-setting, I receive an error:
>> url="http://coastwatch.pfeg.noaa.gov/erddap/griddap/hawaii_9d34_0512_2367.nc?temp[(1950-01-15T00:00:00Z):1:(2008-12-15T00:00:00Z)][(-5.0):1:(-5374.0)][(25):1:(50)][(230):1:(250)],salt[(1950-01-15T00:00:00Z):1:(2008-12-15T00:00:00Z)][(-5.0):1:(-5374.0)][( 25):1:(50 )][(230):1:(250)],u[(1950-01-15T00:00:00Z):1:(2008-12-15T00:00:00Z)][(-5.0):1:(-5374.0)][(25):1:(50)][(230):1:(250 )],v[(195 0-01-15T00:00:00Z):1:(2008-12-15T00:00:00Z)][(-5.0):1:(-5374.0)][(25):1:(50)][(230):1:(250)],w[(1950-01-15T00:00:00Z):1:(2008-12-15T00:00:00Z)][(-5.0):1:(-5374.0)][(25):1:(50)][(230):1:(250)]"
>>
>> filename = url
>> exists = isfilepresent(filename)
>> if(.not.exists) then
>> print("OPeNDAP isfilepresent test unsuccessful.")
>> print("Either file doesn't exist, or NCL does not have OPeNDAP capabilities on this system")
>> else
>> print("OPeNDAP isfilepresent test successful.")
>> f = addfile(filename,"r")
>> vars = getfilevarnames(f)
>> print(vars)
>> end if
>>
>> The error gives me:
>> ncl 48> filename = url
>> ncl 49> exists = isfilepresent(filename)
>> ncopen: filename "http://coastwatch.pfeg.noaa.gov/erddap/griddap/hawaii_9d34_0512_2367.nc?temp[(1950-01-15T00:00:00Z):1:(2008-12-15T00:00:00Z)][(-5.0):1:(-5374.0)][(25):1:(50)][(230):1:(250)],salt[(1950-01-15T00:00:00Z):1:(2008-12-15T00:00:00Z)][(-5.0):1:(-5374.0)][( 25):1:(50 )][(230):1:(250)],u[(1950-01-15T00:00:00Z):1:(2008-12-15T00:00:00Z)][(-5.0):1:(-5374.0)][(25):1:(50)][(230):1:(2 50)],v[(1 950-01-15T00:00:00Z):1:(2008-12-15T00:00:00Z)][(-5.0):1:(-5374.0)][(25):1:(50)][(230):1:(250)],w[(1950-01-15T00:00:00Z):1:(2008-12-15T00:00:00Z)][(-5.0):1:(-5374.0)][(25):1:(50)][(230):1:(250)]": Operation not permitted
>>
>> So I thought the time format was likely to be wrong because when I enter the URL with subset options into a browser, I get an error indicating that the string is converting to a meaningless number with regard to the string time origin:
>> Your query produced no matching results. Query error: For variable=temp axis#0=time Constraint="[(1950-01-15T00:00:00Z):1:(2008-12-15T00:00:00Z)]": Start="-6.299424E8" is less than the axis minimum=-3.774816E8 (and even -3.7879647790507364E8).
>> .
>>
>> So I tried just getting a single time step:
>>
>> url="http://coastwatch.pfeg.noaa.gov/erddap/gridda/hawaii_9d34_0512_2367.nc?temp[(1):1:(2)][(-5.0):1:(-5374.0)][(25):1:(50)][(230):1:(250)]"
>>
>> which still returns an error:
>> ncopen: filename "http://coastwatch.pfeg.noaa.gov/erddap/griddap/hawaii_9d34_0512_2367.nc?temp[(1):1:(2)][(-5.0):1:(-5374.0)][(25):1:(50)][(230):1:(250)]": Operation not permitted
>>
>> OK, so can anyone point me to documentation for the correct subset format to use, please?
>>
>> Best fishes
>>
>> Sam
>> --
>> Sam McClatchie,
>> Supervisory oceanographer, Fisheries oceanography
>> Southwest Fisheries Science Center, NOAA,
>> 8604 La Jolla Shores Drive
>> La Jolla, CA 92037-1508, U.S.A.
>> email <Sam.McClatchie@noaa.gov>
>> Cellular: 858 752 8495
>> Research home page <www.fishocean.info>
>>
>> /\
>> ...>><xX(¡>
>> //// \\\\
>> <¡)Xx><<
>> ///// \\\\\\
>> ><(((¡>
>> >><(((¡> ...>><xX(¡>O<¡)Xx><<
>>
>> _______________________________________________
>> ncl-talk mailing list
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
--
Sam McClatchie,
Supervisory oceanographer, Fisheries oceanography
Southwest Fisheries Science Center, NOAA,
8604 La Jolla Shores Drive
La Jolla, CA 92037-1508, U.S.A.
email <Sam.McClatchie@noaa.gov>
Cellular: 858 752 8495
Research home page <www.fishocean.info>
/\
...>><xX(¡>
//// \\\\
<¡)Xx><<
///// \\\\\\
><(((¡>
>><(((¡> ...>><xX(¡>O<¡)Xx><<
_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Wed Apr 3 12:12:48 2013
This archive was generated by hypermail 2.1.8 : Thu Apr 04 2013 - 21:06:41 MDT