Re: Limit of number of variables

From: David Brown <dbrown_at_nyahnyahspammersnyahnyah>
Date: Fri Feb 25 2011 - 11:49:54 MST

Hi Yaosheng,

Good question. Actually if you are compiling from source it is very easy to increase the number of variables allowed in a file by changing a single line in the header file "defs.h":

#define NCL_MAX_FVARS 2048

You can set it to whatever you like.

The reason that we do not do this in a released version is that there are a number of fixed size arrays that get allocated based on this value whenever a file record is created. It is not that big a deal if you open only one or a few files, but using addfiles to open, say, a 1000 files can result in tying up a couple of hundreds of megabytes of memory. If we quadrupled the number of variables allowed it would be getting close to a gigabyte of memory (mostly wasted in a typical situation where each file contains on the order of 10's or 100's of variables). We realize this is not scalable and we need to make the file record structures dynamic, allowing for a basically unlimited number of variables in a file and tailoring the amount of space consumed by the file record to the number of variables actually found in the file. Until recently it has not been a major issue, because files with large numbers of variables were pretty rare. Nowadays it seems to be more common, so this i
 s an issue that has become high priority for us.

On Feb 24, 2011, at 5:02 PM, Yaosheng Chen wrote:

> Hi David,
> Got it. Thank you.
> Just wonder is there any particular reason that NCL sets this limit?
> Or is it something that we can simply modify the source code to
> remove?
> Thanks.
> Yaosheng
> On Thu, Feb 24, 2011 at 6:59 PM, David Brown <> wrote:
>> Hi Yaosheng,
>> You are right that NCL currently has a limit of 2048 variables. We have a ticket filed for this issue and hope to address it very soon.
>> I'm sorry that we do not have an immediate fix.
>> -dave
>> On Feb 19, 2011, at 8:59 AM, Yaosheng Chen wrote:
>>> Hi Sir/Madam,
>>> I got an error msg when I open a netcdf file in ncl, it says:
>>> The file contains (5323) variable which exceeds the number of
>>> allowable variables (2048),
>>> Is there any way I can increase this limit?
>>> Thanks.
>>> Yaosheng
>>> _______________________________________________
>>> ncl-talk mailing list
>>> List instructions, subscriber options, unsubscribe:
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:

ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
Received on Fri Feb 25 11:49:59 2011

This archive was generated by hypermail 2.1.8 : Wed Mar 02 2011 - 09:18:13 MST