Re: data size issue

Hi Sanjiv,

Ditto here on the late reply. I was in a workshop last week.

The error you're getting looks like it's from running on a 32-bit machine. Is that the case? If so, there are restrictions on 32-bit systems that limits you to 2 GB variables.

Please see our FAQ at:

http://www.ncl.ucar.edu/FAQ/#mem_issues_001

--Mary

On Jan 15, 2014, at 4:23 PM, Sanjiv Kumar <skumar.water@gmail.com> wrote:

> Hi Mary:
>
> Sorry for the late reply. Below is the error message i get when i run my script on COLA machines:
>
> ************************************************************************
>
> fatal:Aggregating variable trefmxav from file list variable f1 as specified would exceed maximum NCL variable size
> fatal:["Execute.c":8128]:Execute: Error occurred at or near line 46 in file VAR_ANA_TREFMXAV_MAYICS.ncl
>
> fatal:Variable (var1SP) is undefined
> fatal:["Execute.c":8128]:Execute: Error occurred at or near line 48 in file VAR_ANA_TREFMXAV_MAYICS.ncl
>
> fatal:Aggregating variable trefmxav from file list variable f2 as specified would exceed maximum NCL variable size
> fatal:["Execute.c":8128]:Execute: Error occurred at or near line 61 in file VAR_ANA_TREFMXAV_MAYICS.ncl
> ******************************************************************************
>
>
> However, when i run the same script on 'geyser', it runs fine. Both machines have same NCL version 6.1.2
>
> Let me know if any further details is needed. This time i will reply early.
>
> Best Regards
> Sanjiv Kumar
>
>
> On Thu, Jan 9, 2014 at 10:28 AM, Mary Haley <haley@ucar.edu> wrote:
> Hi Sanjiv,
>
> What do you mean by "it breaks down"? Do you actually get an error message, or does the script take a long time to run?
>
> Also, when you say "large data volume", do you mean you have a few really large variables, or several variables that take up more than 2 GB?
>
> It would help if we could see a script that illustrates the problem, and if we had an idea of the size of your variables (you can get this by using printVarSummary on your variables).
>
> If you are not sure where it is "breaking down", then you can try commenting out the "begin" and "end" statements in your main code (if any), and then running the script with:
>
> ncl -x your_script.ncl
>
> The "-x" will cause every line to be echoed as it is executed.
>
> --Mary
>
> On Jan 7, 2014, at 1:22 PM, Sanjiv Kumar <skumar.water@gmail.com> wrote:
>
> > Hello:
> >
> > Happy New Year!
> >
> > I am running a ncl script, that handles large data volume (i guess more than 2GB). The script runs fine when i use NCAR machines (geyser). However, when i run the same script on our own machines (COLA machines) using the same NCL version (6.1.2); it breaks down, most likely due to large data volume size.
> >
> > We are not sure how to fix this problem. Any suggestion/help is highly appreciated.
> >
> > Best Regards
> > Sanjiv Kumar
> >
> >
> > _______________________________________________
> > ncl-talk mailing list
> > List instructions, subscriber options, unsubscribe:
> > http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Tue Jan 21 13:20:48 2014