Re: Limit of number of variables

Hi David,

 I see. Thank you.
Looking forward to new versions.

Have a nice day.
Yaosheng

On Fri, Feb 25, 2011 at 1:49 PM, David Brown <dbrown@ucar.edu> wrote:
> Hi Yaosheng,
>
> Good question. Actually if you are compiling from source it is very easy to increase the number of variables allowed in a file by changing a single line in the header file "defs.h":
>
> #define NCL_MAX_FVARS 2048
>
> You can set it to whatever you like.
>
> The reason that we do not do this in a released version is that there are a number of fixed size arrays that get allocated based on this value whenever a file record is created.  It is not that big a deal if you open only one or a few files, but using addfiles to open, say, a 1000 files can result in tying up a couple of hundreds of megabytes of memory. If we quadrupled the number of variables allowed it would be getting close to a gigabyte of memory (mostly wasted in a typical situation where each file contains on the order of 10's or 100's of variables).  We realize this is not scalable and we need to make the file record structures dynamic, allowing for a basically unlimited number of variables in a file and tailoring the amount of space consumed by the file record to the number of variables actually found in the file. Until recently it has not been a major issue, because files with large numbers of variables were pretty rare. Nowadays it seems to be more common, so this is an issue that  has become high priority for us.
>  -dave
>
>
> On Feb 24, 2011, at 5:02 PM, Yaosheng Chen wrote:
>
>> Hi David,
>>
>> Got it. Thank you.
>>
>> Just wonder is there any particular reason that NCL sets this limit?
>> Or is it something that we can simply modify the source code to
>> remove?
>>
>> Thanks.
>> Yaosheng
>>
>> On Thu, Feb 24, 2011 at 6:59 PM, David Brown <dbrown@ucar.edu> wrote:
>>> Hi Yaosheng,
>>>
>>> You are right that NCL currently has a limit of 2048 variables. We have a ticket filed for this issue and hope to address it very soon.
>>> I'm sorry that we do not have an immediate fix.
>>>  -dave
>>>
>>> On Feb 19, 2011, at 8:59 AM, Yaosheng Chen wrote:
>>>
>>>> Hi Sir/Madam,
>>>>
>>>> I got an error msg when I open a netcdf file in ncl, it says:
>>>>
>>>> The file contains (5323) variable which  exceeds the number of
>>>> allowable variables (2048),
>>>>
>>>> Is there any way I can increase this limit?
>>>>
>>>> Thanks.
>>>> Yaosheng
>>>> _______________________________________________
>>>> ncl-talk mailing list
>>>> List instructions, subscriber options, unsubscribe:
>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>
>>>
>> _______________________________________________
>> ncl-talk mailing list
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Fri Feb 25 12:06:44 2011