Re: data size issue

Thanks Dave.

I am getting answer 4. So, my system administrator has to fix it.

-Sanjiv Kumar

On Wed, Jan 22, 2014 at 4:40 PM, David Brown <dbrown@ucar.edu> wrote:

> Sanjiv,
> This message is generated in the variable aggregating routine where the
> number of bytes in the requested aggregation is compared with the biggest
> number that can be understood by the allocation routine.
> This depends on the size of the size_t type: 4 bytes or 8 bytes. If size_t
> is 4 bytes the biggest possible number is 2147483647; if 8 bytes it's a
> very large number > 9e18. In practice, this message should only ever be
> encountered when running in 32 bit mode where size_t is 4 bytes. Note
> that you can run 32-bit programs on most 64-bit systems, so it is possible
> that you are running a 32-bit version of NCL. You should quickly be able to
> tell whether you are running a 32-bit version of NCL: just execute this
> statement:
>
> ncl 1> print(sizeof("long"))
>
> If you get the answer 4, then you have a 32-bit version of NCL. If you get
> the answer 8, then something is probably wrong. You might want to try to
> estimate the size of the aggregated variable you are reading. But we may
> have to take a look.
> -dave
>
>
>
>
>
> On Jan 22, 2014, at 3:14 PM, Mary Haley <haley@ucar.edu> wrote:
>
> > Sanjiv,
> >
> > Are you running with at least version 6.0.0 of NCL? What does "ncl -V"
> report?
> >
> > Perhaps another possibility is how many files you are passing to
> "addfiles". I'm hoping Dave Brown will weigh in here.
> >
> > Dave, do you know what this error message signifies:
> >
> >>> fatal:Aggregating variable trefmxav from file list variable f1 as
> specified would exceed maximum NCL variable size
> >
> > --Mary
> >
> > On Jan 22, 2014, at 2:50 PM, Sanjiv Kumar <skumar.water@gmail.com>
> wrote:
> >
> >> Hi Mary:
> >>
> >> COLA system administrator has informed me that COLA machines are 64-bit
> machines, i.e. I am trying to run the program on 64-bit machines.
> >>
> >> -Sanjiv Kumar
> >>
> >>
> >>
> >>
> >> On Tue, Jan 21, 2014 at 1:20 PM, Mary Haley <haley@ucar.edu> wrote:
> >> Hi Sanjiv,
> >>
> >> Ditto here on the late reply. I was in a workshop last week.
> >>
> >> The error you're getting looks like it's from running on a 32-bit
> machine. Is that the case? If so, there are restrictions on 32-bit systems
> that limits you to 2 GB variables.
> >>
> >> Please see our FAQ at:
> >>
> >> http://www.ncl.ucar.edu/FAQ/#mem_issues_001
> >>
> >> --Mary
> >>
> >> On Jan 15, 2014, at 4:23 PM, Sanjiv Kumar <skumar.water@gmail.com>
> wrote:
> >>
> >>> Hi Mary:
> >>>
> >>> Sorry for the late reply. Below is the error message i get when i run
> my script on COLA machines:
> >>>
> >>>
> ************************************************************************
> >>>
> >>> fatal:Aggregating variable trefmxav from file list variable f1 as
> specified would exceed maximum NCL variable size
> >>> fatal:["Execute.c":8128]:Execute: Error occurred at or near line 46 in
> file VAR_ANA_TREFMXAV_MAYICS.ncl
> >>>
> >>> fatal:Variable (var1SP) is undefined
> >>> fatal:["Execute.c":8128]:Execute: Error occurred at or near line 48 in
> file VAR_ANA_TREFMXAV_MAYICS.ncl
> >>>
> >>> fatal:Aggregating variable trefmxav from file list variable f2 as
> specified would exceed maximum NCL variable size
> >>> fatal:["Execute.c":8128]:Execute: Error occurred at or near line 61 in
> file VAR_ANA_TREFMXAV_MAYICS.ncl
> >>>
> ******************************************************************************
> >>>
> >>>
> >>> However, when i run the same script on 'geyser', it runs fine. Both
> machines have same NCL version 6.1.2
> >>>
> >>> Let me know if any further details is needed. This time i will reply
> early.
> >>>
> >>> Best Regards
> >>> Sanjiv Kumar
> >>>
> >>>
> >>> On Thu, Jan 9, 2014 at 10:28 AM, Mary Haley <haley@ucar.edu> wrote:
> >>> Hi Sanjiv,
> >>>
> >>> What do you mean by "it breaks down"? Do you actually get an error
> message, or does the script take a long time to run?
> >>>
> >>> Also, when you say "large data volume", do you mean you have a few
> really large variables, or several variables that take up more than 2 GB?
> >>>
> >>> It would help if we could see a script that illustrates the problem,
> and if we had an idea of the size of your variables (you can get this by
> using printVarSummary on your variables).
> >>>
> >>> If you are not sure where it is "breaking down", then you can try
> commenting out the "begin" and "end" statements in your main code (if any),
> and then running the script with:
> >>>
> >>> ncl -x your_script.ncl
> >>>
> >>> The "-x" will cause every line to be echoed as it is executed.
> >>>
> >>> --Mary
> >>>
> >>> On Jan 7, 2014, at 1:22 PM, Sanjiv Kumar <skumar.water@gmail.com>
> wrote:
> >>>
> >>>> Hello:
> >>>>
> >>>> Happy New Year!
> >>>>
> >>>> I am running a ncl script, that handles large data volume (i guess
> more than 2GB). The script runs fine when i use NCAR machines (geyser).
> However, when i run the same script on our own machines (COLA machines)
> using the same NCL version (6.1.2); it breaks down, most likely due to
> large data volume size.
> >>>>
> >>>> We are not sure how to fix this problem. Any suggestion/help is
> highly appreciated.
> >>>>
> >>>> Best Regards
> >>>> Sanjiv Kumar
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> ncl-talk mailing list
> >>>> List instructions, subscriber options, unsubscribe:
> >>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
> >>>
> >>>
> >>> _______________________________________________
> >>> ncl-talk mailing list
> >>> List instructions, subscriber options, unsubscribe:
> >>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
> >>
> >>
> >> _______________________________________________
> >> ncl-talk mailing list
> >> List instructions, subscriber options, unsubscribe:
> >> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
> >
> > _______________________________________________
> > ncl-talk mailing list
> > List instructions, subscriber options, unsubscribe:
> > http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Fri Jan 24 13:18:57 2014