Re: (no subject)

Hi Gang,

You can avoid running out of file descriptors, which on typical Unix/
Linux systems is limited to 1024, by making the following call prior
to your call to 'addfiles':

setfileoption("nc","SuppressClose", False)

This will cause NCL to close the data files after each access and
thus limits the number of simultaneously opened file descriptors open
to just a few.

This mode can be inefficient when files are open for writing, which
is why it is not the default behavior, but it does not affect
performance much for read-only access. We are considering having
different default behaviors for read vs. write access. It has not
been implemented yet because it requires new behavior from the
code that underlies the 'setfileoption' functionality.

Hope this helps.
  -dave

On Dec 5, 2008, at 4:02 AM, Gang Liu wrote:

> Hello,
>
> My intention is to obtain time series analysis results of some
> variables from the WRF model output during the last half-hour of
> integral. I set the output interval as 1 second, so there are 1800
> times of output results in total during the integral of half hour.
> Previously I had them written in one file. The NCL was not able to
> read variables out of the mega file close to 300 Gbytes in size
> encountering memory-related problem.
>
> Now I have the output results written in 1800 files, i.e. one file
> for one output of 1 second. So every file is less than 0.5 Gbytes
> in size now.
>
> I write the NCL code as follows:
>
> ----------------------------------------------------------------------
> ---------------------------------
>
> begin
>
> fils=systemfunc("ls wrfout_d01_0001-01-01_03:*")
> f=addfiles(fils+".nc","r")
> ListSetType(f,"join")
>
> u0=f[:]->U
> v0=f[:]->V
> w0=f[:]->W
>
> th00=f[:]->T
> th0=th00+300.0
>
> tke0=f[:]->TKE
> xkmv0=f[:]->XKMV
> xkhv0=f[:]->XKHV
>
> ph0=f[:]->PH
> phb0=f[:]->PHB
>
> ……………………………………………………………
>
> ----------------------------------------------------------------------
> ---------------------------------
>
> However I got the error message as follows:
>
> ----------------------------------------------------------------------
> ---------------------------------
>
> Copyright (C) 1995-2007 - All Rights Reserved
> University Corporation for Atmospheric Research
> NCAR Command Language Version 5.0.0
> The use of this software is governed by a License Agreement.
> See http://www.ncl.ucar.edu/ for more details.
> fatal:Too many open files
> fatal:Could not open (wrfout_d01_0001-01-01_03:13:10)
> fatal:Too many open files
> fatal:Could not open (wrfout_d01_0001-01-01_03:13:09)
> fatal:Too many open files
> fatal:Could not open (wrfout_d01_0001-01-01_03:13:08)
> fatal:Too many open files
> fatal:Could not open (wrfout_d01_0001-01-01_03:13:07)
> fatal:Too many open files
> fatal:Could not open (wrfout_d01_0001-01-01_03:13:06)
> ……………………………………………………………
> ……………………………………………………………
> ……………………………………………………………
> fatal:Too many open files
> fatal:Could not open (wrfout_d01_0001-01-01_03:00:02)
> fatal:Too many open files
> fatal:Could not open (wrfout_d01_0001-01-01_03:00:01)
>
> ----------------------------------------------------------------------
> ---------------------------------
>
> I have uploaded the NCL script file (nclpost.ncl) and one of the
> data files (wrfout_d01_0001-01-01_03:00:01) to the directory
> ‘incoming’ in ftp.cgd.ucar.edu.
>
> Thanks a lot.
>
> Gang
>
> ************************************
> Gang Liu
> Department of Atmospheric Sciences
> Nanjing University
> Nanjing 210093
> P. R. China
> Phone: 86-25-83595185 (o)
> Email: gangliu_at_nju.edu.cn
> ************************************
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Fri Dec 05 2008 - 16:33:36 MST