Re: segmentation fault (core dumped) appeared when processing a huge size.

Do you have ideas about your machine memory?
If your machine does not have 16G memory,
you'd better do it one by one on time level.

You may also consider in-sito transform,
instead of create a new file, especially when it is a large file.

Wei

huangwei@ucar.edu
VETS/CISL
National Center for Atmospheric Research
P.O. Box 3000 (1850 Table Mesa Dr.)
Boulder, CO 80307-3000 USA
(303) 497-8924

On Mar 2, 2014, at 11:24 PM, dyjbean@gmail.com wrote:

>
> hi,
> when processing a huge size netcdf file, " segmentation faule (core dumped) " appeared.
>
> my file information is as below:
> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ncdump -h MODIS17A2_2000-2012_GPP.nc
> netcdf MODIS17A2_2000-2012_GPP {
> dimensions:
> time = 156 ;
> lat = 3600 ;
> lon = 7200 ;
> variables:
> int time(time) ;
> time:long_name = "time" ;
> time:units = "yyyymm" ;
> float lat(lat) ;
> lat:long_name = "latitude" ;
> lat:units = "degrees_north" ;
> float lon(lon) ;
> lon:long_name = "longitude" ;
> lon:units = "degrees_east" ;
> float gpp(time, lat, lon) ;
> gpp:long_name = "Gross Primary Production(GPP)" ;
> gpp:units = "KgC/m2" ;
> gpp:_FillValue = -9999.f ;
> }
>
> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
> the file size is 16174124456, about 16 G. i want to convert its dimensionality (time,lat,lon) to (lat,lon,time) for further EOF analysis.
>
> my script is :
> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"
> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"
> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"
> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/shea_util.ncl"
>
> begin
> infile="eof_gpp/MODIS17A2_2000-2012_GPP.nc"
> outfile="MODIS17A2_2000-2012_GPP_time.nc"
>
> setfileoption("nc","Format","LargeFile")
> ff=addfile(infile,"r")
> lat=ff->lat
> lon=ff->lon
> time=ff->time
> sgpp=ff->gpp
>
> dims=dimsizes(sgpp)
> gpp=new((/dims(1),dims(2),dims(0)/),"float",-9999.)
>
> ncdf=addfile(outfile,"c")
> ;delete(ff)
>
> ncdf->time=time
> ncdf->lat=lat
> ncdf->lon=lon
> ncdf->gpp=sgpp(lat|:,lon|:,time|:)
>
> delete(ff)
> delete(lat)
> delete(lon)
> delete(time)
> delete(sgpp)
> delete(gpp)
> delete(ncdf)
> end
>
> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
> but "segmentation faule (core dumped)" appeared, how can i do this conversion?
>
> thanks
>
> dyjbean@gmail.com
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Mon Mar 3 11:16:33 2014