Re: too many open files error

From: Wei Huang <huangwei_at_nyahnyahspammersnyahnyah>
Date: Mon Nov 04 2013 - 21:32:28 MST

Lijun,

If your wrf files are in netcdf 3 (try "ncdump -k one-of-your-wrf-file"),
you may refer to:

http://www.ncl.ucar.edu/Support/talk_archives/2013/1637.html

If your wrf files are netcdf4, please check:

http://www.ncl.ucar.edu/Support/talk_archives/2013/1995.html

For such problem, you may google "too many open files"
from NCL web-site, which will check through NCL's archive
emails, which, lots of times, will give you some help.

Wei

huangwei@ucar.edu
VETS/CISL
National Center for Atmospheric Research
P.O. Box 3000 (1850 Table Mesa Dr.)
Boulder, CO 80307-3000 USA
(303) 497-8924

On Nov 4, 2013, at 7:34 PM, Lijun Diao <ljdiao@gmail.com> wrote:

> Dear NCL talk:
>
> I am trying to read hundreds of wrf output files as the code below:
>
> standard_dir = "/dir1//STANDARD/"
> biomass_dir = "/dir2/NO_BIOMASS/"
>
> standard_FILES = systemfunc( "ls -1 " + standard_dir + "wrfout_d01*" )
> biomass_FILES = systemfunc( "ls -1 " + biomass_dir + "wrfout_d01*" )
>
> standard_out = addfiles( standard_FILES + ".nc", "r" )
> biomass_out = addfiles( biomass_FILES + ".nc", "r" )
>
> standard_times = wrf_user_getvar( standard_out, "times", -1 )
> standard_ntimes = dimsizes( standard_times )
>
> There are two directories, standard_dir and biomass_dir, and each contains 720 hourly files. In futher, the file can be 6 times more. The extract of ncl running errors are:
>
> #001: H5F.c line 1211 in H5F_open(): unable to open file: time = Tue Nov 5 02:25:29 2013
> , name = '/tng1/users/ldiao/WRF-Chem_NARR/WRFV3/run/NO_BIOMASS/wrfout_d01_2012-05-01_00:00:00', tent_flags = 0
> major: File accessability
> minor: Unable to open file
> #002: H5FD.c line 1086 in H5FD_open(): open failed
> major: Virtual File Layer
> minor: Unable to initialize object
> #003: H5FDsec2.c line 362 in H5FD_sec2_open(): unable to open file: name = '/tng1/users/ldiao/WRF-Chem_NARR/WRFV3/run/NO_BIOMASS/wrfout_d01_2012-05-01_00:00:00', errno = 24, error message = 'Too many open files', flags = 0, o_flags = 0
> major: File accessability
> minor: Unable to open file
> Unable to open file: </tng1/users/ldiao/WRF-Chem_NARR/WRFV3/run/NO_BIOMASS/wrfout_d01_2012-05-01_00:00:00>, at line: 32
> fatal:NclHDF5: Unable to open file: </tng1/users/ldiao/WRF-Chem_NARR/WRFV3/run/NO_BIOMASS/wrfout_d01_2012-05-01_00:00:00>, at line: 27831272
>
> it seems the error is caused by too many open files. So how can I conquer this problem? I know I can make the several wrf output files into one file.
>
> thanks,
> -----------------------------------------------------------------------------
> Lijun Diao
> ldiao@uh.edu
> Research Assistant
> Department of Earth & Atmospheric Sciences
> University of Houston
> 4800 Calhoun RD., Bldg. S&R1-312
> Houston, TX 77204-5007
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Mon Nov 4 21:33:53 2013

This archive was generated by hypermail 2.1.8 : Mon Nov 11 2013 - 09:45:33 MST