Hi DJ,
When you get a NclMalloc error it generally
means you are running out of memory, as Wei
alluded to. You might try using the netCDF
operator ncra:
http://nco.sourceforge.net/nco.html#ncra-netCDF-Record-Averager
This allows you to create an average outside of
the NCL framework, although I seriously doubt
that NCL is the cause of your memory issues. You
will likely have to split your data up, as your
array size sounds like it is too big for the
machine you are working on. (Also note that NCL
can not handle an array > 2GB.)
Best regards,
Adam
DJ Rasmussen wrote:
> NCL Community,
>
> I have 14 u and v wind component CRCM files that I would
> like to average. I have been trying to create a script
> that reads the files in and performs one giant average of
> all of the data, but I am running into memory errors. Even
> averaging just one file alone gives an error with this
> script:
>
> Error: fatal:NclMalloc Failed:[errno=12]
>
> Script:
> ;*************************************************
> ;NARCCAPAVG.ncl
> ;
> ;************************************************
> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"
> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"
> load
> "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"
>
> ;************************************************
> begin
>
> ;************************************************
> ; read in multiple netCDF files
> ;************************************************
> diri = "/home/rasmussen/NARCCAP/netCDF/RCM3/" ; directory
> where files reside
> fils = systemfunc("ls "+diri+"uas_CRCM_*.nc")
> nfil = dimsizes(fils)
> print(fils)
>
> ;*************************Get Variables**********
> f = addfiles (fils, "r")
>
> UAS = f[:]->uas ; (time,yc,xc)
>
> printVarSummary(UAS)
>
> ;*************************Calculate Average******
>
> WSPD_AVG = dim_avg_n(UAS,0) ; (lat,lon)
> WSPD_AVG_at_long_name = "mean wind speed"
>
> printVarSummary(WSPD_AVG)
>
> ;************************************************
> ; output ascii
> ;************************************************
>
> fName = "RCM3AVG.txt"
> system("/bin/rm -f "+fName) ; remove if file is present
> asciiwrite (fName , WSPD_AVG)
>
> end
>
> The basic script can read smaller files with less time
> steps just fine (8760 steps), but it gives an error with
> the files that have 14600 time steps. (3hrly over 5 years)
>
> Variable: UAS
> Type: float
> Total Size: 940240000 bytes
> 235060000 values
> Number of Dimensions: 3
> Dimensions and sizes: [time | 14600] x [yc | 115] x [xc |
> 140]
> Coordinates:
> time: [8395.125..10220]
> yc: [ 0..5700000]
> xc: [ 0..6950000]
> Number Of Attributes: 13
> units : m s-1
> type : instantaneaous
> long_name : Zonal Surface Wind Speed
> standard_name : eastward_wind
> missing_value : 1e+20
> _FillValue : 1e+20
> actual_range : ( -41.18215, 38.8077 )
> add_offset : 0
> scale_factor : 1
> coordinates : lon lat level
> grid_mapping : polar_stereographic
> level_desc : 10 m
> grid_desc : polar_stereographic
>
> I am at a loss to explain why this is happening, but I am
> going to have to resolve this problem before I can
> continue trying to be able to perform heavier computations
> with 14 files.
>
> Is there a way that this
> http://www.ncl.ucar.edu/Applications/Scripts/crcm_1.ncl
> script can be modified to accommodate what I am trying to
> be able to do? This might be the easier route for me since
> my FORTRAN abilities are not so stellar.
>
> Any advice is appreciated. This doesn't seem like a
> complicated problem compared to the other issues that are
> submitted to NCL talk.
>
> Kind regards,
>
> DJ Rasmussen
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
-- __________________________________________________ Adam Phillips asphilli_at_ucar.edu National Center for Atmospheric Research tel: (303) 497-1726 Climate and Global Dynamics Division fax: (303) 497-1333 P.O. Box 3000 Boulder, CO 80307-3000 http://www.cgd.ucar.edu/cas/asphilli _______________________________________________ ncl-talk mailing list List instructions, subscriber options, unsubscribe: http://mailman.ucar.edu/mailman/listinfo/ncl-talkReceived on Mon Oct 05 2009 - 11:14:44 MDT
This archive was generated by hypermail 2.2.0 : Mon Oct 05 2009 - 13:28:34 MDT