Re: memory allocation

Noel,

You said that the 320gb memory is over 5 nodes.
I guess you have a Linux Cluster, if that is the case,
NCL can only use the memory from one node,
which means, about 64gb.

Lots of times, you may not need such a big array
for your data, as you may only process one part
of the array. In which case, you may only need
a portion of the memory allocated first.

Again, please give us more details, and we'll try to help.

Wei

huangwei@ucar.edu
VETS/CISL
National Center for Atmospheric Research
P.O. Box 3000 (1850 Table Mesa Dr.)
Boulder, CO 80307-3000 USA
(303) 497-8924

On Nov 5, 2012, at 12:24 PM, Noel Aloysius wrote:

> Hi,
>
> I am using NCL 6.1.0. The system is Redhat Enterprise 64-bit and I have 320gb exclusively allocated over 5 nodes.
>
> I am able to create one array that takes 40gb. The Malloc error is thrown when I try to initialize the second array.
>
> Thanks,
> Noel
>
>
> Noel Aloysius
>
>
> On Mon, Nov 5, 2012 at 12:35 PM, Wei Huang <huangwei@ucar.edu> wrote:
> Noel,
>
> First, to use large memory you need NCL 6.0 or later, and make sure it is 64bit built.
> Second, your machine has 320gb, does not mean mean each individual user can use that amount.
> You need to check with your system admin that you can use up to that amount of memory.
>
> Any jobs manipulate that amount of data/memory will be slow, and very slow in I/O,
> hope you have already thought of this.
>
> Please give us more details, and we will try to help you.
>
> Wei
>
> huangwei@ucar.edu
> VETS/CISL
> National Center for Atmospheric Research
> P.O. Box 3000 (1850 Table Mesa Dr.)
> Boulder, CO 80307-3000 USA
> (303) 497-8924
>
>
>
>
>
>
> On Nov 5, 2012, at 10:17 AM, Noel Aloysius wrote:
>
>> Hi NCL-talk,
>>
>> I want to load two datasets that will take a total of 90gb memory and do some processing. The total memory available is 320gb.
>>
>> Is it possible to do this. I tried but got the NclMalloc error.
>>
>> Thanks,
>> Noel
>>
>>
>>
>> Noel Aloysius
>> _______________________________________________
>> ncl-talk mailing list
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Mon Nov 5 14:20:39 2012