Simultaneously opening many NCL files

From: David Brown <dbrown_at_nyahnyahspammersnyahnyah>
Date: Fri, 14 Sep 2007 16:19:12 -0600

Hi all,

With the newest version of NCL we made the decision to change the
default for the NetCDF file option 'SuppressClose'
from True to False. This changes NCL's behavior such that instead of
closing a file after each operation it keeps the
file open until the user (1) deletes the file variable (2) reassigns
the file variable or (3) exits from NCL.
This can help to dramatically improve NCL's efficiency particularly
when writing large files with many variables,
and, since we had never noticed any adverse effects, we saw no harm
it making it the default.

However, we have recently encountered situations where there is a
drawback to keeping files open in this manner.
Users whose scripts open many files simultaneously (using 'addfiles'
or 'addfile' in a loop) may find they fail with
NCL 4.3.1. This is because they are bumping up against limits imposed
by the operating system on the number of
file descriptors available to a single process. Although the limit
varies, it is often set to 1024.

If you have a script that opens many files at once, that worked prior
to 4.3.1, and seems to be having problems now,
you should be able to fix it by adding, prior to your calls to
'addfile' or 'addfiles' , this call:

setfileoption("nc","SuppressClose", False)

We are sorry for any inconvenience resulting from this change.
   -dave

_______________________________________________
ncl-talk mailing list
ncl-talk_at_ucar.edu
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Fri Sep 14 2007 - 16:19:12 MDT

This archive was generated by hypermail 2.2.0 : Mon Sep 17 2007 - 14:58:21 MDT