Re: how to extract data from a huge nc data file?

From: Xianglei Huang <xianglei_at_nyahnyahspammersnyahnyah>
Date: Mon, 23 Jun 2008 14:28:44 -0400

        ncks is the right command for this type of job. It seems to me that
most likely you are using a 32-bit OS so it cannot handle netcdf files
beyond 4G (or 2G?). Try it on a 64-bit machine and it should work.

        
Zhang Tao wrote:
> Hi, all,
>
> I have a huge netcdf data file
> "thetao_O1.20C3M_1.CCSM.ocnm.1950-01_cat_1999-12.nc" and
> the size is about 12G.
>
> I want to extract parts of data and generate a small size netcdf file,
> the specific command is listed in point 3 below
> and I get the error message "segmentation fault" see point 2 below.
> I can use "ncdump" to look at the attribution of the data, see point
> 1 below.
>
> Does anyone know how to extract data from this huge nc file?
>
> Thanks,
> Tao
>
>
>
> 1)
> netcdf thetao_O1.20C3M_1.CCSM.ocnm.1950-01_cat_1999-12 {
> dimensions:
> lon = 320 ;
> lat = 395 ;
> depth = 40 ;
> bnds = 2 ;
> time = UNLIMITED ; // (600 currently)
> variables:
> double lon_bnds(lon, bnds) ;
> double lat_bnds(lat, bnds) ;
> double depth_bnds(depth, bnds) ;
> double time_bnds(time, bnds) ;
> double lon(lon) ;
> lon:axis = "X" ;
> lon:standard_name = "longitude" ;
> lon:bounds = "lon_bnds" ;
> lon:long_name = "Longitude" ;
> lon:units = "degrees_east" ;
> double lat(lat) ;
> lat:axis = "Y" ;
> lat:standard_name = "latitude" ;
> lat:bounds = "lat_bnds" ;
> lat:long_name = "Latitude" ;
> lat:units = "degrees_north" ;
> double depth(depth) ;
> depth:axis = "Z" ;
> depth:standard_name = "depth" ;
> depth:positive = "down" ;
> depth:units = "m" ;
> depth:bounds = "depth_bnds" ;
> double time(time) ;
> time:calendar = "noleap" ;
> time:standard_name = "time" ;
> time:axis = "T" ;
> time:units = "days since 0000-1-1" ;
> time:bounds = "time_bnds" ;
> float thetao(time, depth, lat, lon) ;
> thetao:comment = "Created using NCL code
> CCSM_ocnm_2cf.ncl on\n",
> " machine mineral" ;
> thetao:missing_value = 1.e+20f ;
> thetao:long_name = "sea_water_potential_temperature" ;
> thetao:cell_methods = "time: mean (interval: 1 month)" ;
> thetao:history = "Interpolated to regular grid from
> dipole grid,\n",
> "TEMP+273.15" ;
> thetao:units = "K" ;
> thetao:original_units = "C" ;
> thetao:original_name = "TEMP" ;
> thetao:standard_name = "sea_water_potential_temperature" ;
> thetao:_FillValue = 1.e+20f ;
>
> 2)
> ---------------------------
> /nas/tzhang/ZTIPCC4/ncar_ccsm3_0/20c3m/nc>
> extract_thetao_20C3M_1.CCSM.csh &
> [1] 41886
> /nas/tzhang/ZTIPCC4/ncar_ccsm3_0/20c3m/nc> Segmentation fault
>
> [1] Exit 139 extract_thetao_20C3M_1.CCSM.csh
> ----------------------------------------------------
>
> 3)
> ncks -c -F -d depth,1,17 -v thetao
> thetao_O1.20C3M_1.CCSM.ocnm.1950-01_cat_1999-12.nc
> thetao_upper_O1.20C3M_1.CCSM.ocnm.1950-01_cat_1999-12.nc
>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk_at_ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>

_______________________________________________
ncl-talk mailing list
ncl-talk_at_ucar.edu
http://mailman.ucar.edu/mailman/listinfo/ncl-talk

Received on Mon Jun 23 2008 - 12:28:44 MDT

This archive was generated by hypermail 2.2.0 : Wed Jun 25 2008 - 12:04:57 MDT