Re: memory limit

Michael:

you are hitting the 2 GB limit here. I have found that you can slide
by a little past it, differing with different applications. Tiger is
not 64 bit (really), and NCL is compiled in 32 bit mode - so you are
stuck here. Until we get Leopard, and NCL can be made 64 bit, you
will probably have to work out of core, I think.

Sorry.

Lou

On Feb 6, 2007, at 9:28 AM, Michael Notaro wrote:

> Mateus,
>
> Thanks for your suggestion. I tried it, but just get a warning.
>
> [mmia:~/test] notaro% ulimit -s unlimited
> /usr/bin/ulimit: line 4: ulimit: stack size: cannot modify limit:
> Operation not permitted
>
> I tried creating a variable of dimensions (2760,360,720) with both
> NCL and fortran
> on my mac and tempest (ncar) and it always exceeds the memory
> limit, if "float".
>
> Mike
>
> On Feb 6, 2007, at 3:54 AM, Mateus da Silva Teixeira wrote:
>
>> Hi Michael,
>>
>> Have you ever try use 'ulimit -s unlimited'?
>> Some times I couldn't work with large files until do it.
>>
>> Mateus
>>
>> Michael Notaro escreveu:
>>> I am running the following on my Mac OS-X and am hitting a
>>> vm_allocate
>>> memory limit. The problem is that I create 2 large variables,
>>> tasflip(1200,360,720)
>>> and var(1560,360,720). My mac only allows me to create an array
>>> as large
>>> as (2200,360,720), so the code crashes. But I really need to
>>> work with all the data,
>>> so unless there is some trick to get around this problem, I am
>>> stuck. Any tips?
>>>
>>> Thanks,
>>> Michael
>>>
>>> ********************************
>>> core file size (blocks, -c) 0
>>> data seg size (kbytes, -d) 6144
>>> file size (blocks, -f) unlimited
>>> max locked memory (kbytes, -l) unlimited
>>> max memory size (kbytes, -m) unlimited
>>> open files (-n) 256
>>> pipe size (512 bytes, -p) 1
>>> stack size (kbytes, -s) 8192
>>> cpu time (seconds, -t) unlimited
>>> max user processes (-u) 100
>>> virtual memory (kbytes, -v) unlimited
>>> *********************************
>>>
>>> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"
>>> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"
>>> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/shea_util.ncl"
>>> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"
>>> begin
>>>
>>> a = addfile("/Volumes/biggrape/bigmike/
>>> cru_ts_2_10.1901-2002.tmp.nc","r")
>>> tmp = a->tmp
>>> latit = a->lat
>>> longit = a->lon
>>>
>>> climo = clmMonTLL(tmp(828:1187,:,:)) ; 1970-1999 mean used for
>>> climo
>>>
>>> delete(tmp)
>>> delete(a)
>>>
>>> latit!0="lat"
>>> latit_at_units="degrees_north"
>>> latit&lat=latit
>>> longit!0="lon"
>>> longit_at_units="degrees_east"
>>> longit&lon=longit
>>>
>>> b = addfile("/Volumes/ftpstore2/IPCC/sresa2/atm/tas/ncar_ccsm3_0/
>>> run1/tas_A1.nc","r")
>>> lat = doubletofloat(b->lat)
>>> lon = doubletofloat(b->lon)
>>> time = doubletofloat(b->time)
>>> tas = b->tas
>>>
>>> c = addfile("/Volumes/ftpstore2/IPCC/20c3m/atm/tas/ncar_ccsm3_0/
>>> run1/tas_A1.20C3M_1.CCSM.atmm.1980-01_cat_1989-12.nc","r")
>>> d = addfile("/Volumes/ftpstore2/IPCC/20c3m/atm/tas/ncar_ccsm3_0/
>>> run1/tas_A1.20C3M_1.CCSM.atmm.1990-01_cat_1999-12.nc","r")
>>> modern1 = c->tas
>>> modern2 = d->tas
>>> mod1=clmMonTLL(modern1)
>>> mod2=clmMonTLL(modern2)
>>> modern = (mod1+mod2)/2.
>>>
>>> delete(modern1)
>>> delete(modern2)
>>> delete(mod1)
>>> delete(mod2)
>>> delete(c)
>>> delete(d)
>>>
>>> count=0
>>> do iy=0,99
>>> do im=0,11
>>> tas(count,:,:)=tas(count,:,:)-modern(im,:,:)
>>> count=count+1
>>> end do
>>> end do
>>>
>>> delete(modern)
>>>
>>> tasflip=lonFlip(tas)
>>> delete(tas)
>>> var=new((/1560,360,720/),float)
>>>
>>> .....etc...
>>>
>>>
>>> _______________________________________________
>>> ncl-talk mailing list
>>> ncl-talk_at_ucar.edu
>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>
>>
>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk_at_ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk

------------------------------------------------------------------------

----
| Dr. Louis J. Wicker
| NSSL/WRDD
| National Weather Center
| 120 David L. Boren Boulevard, Norman, OK 73072-7323
|
| E-mail:   Louis.Wicker_at_noaa.gov
| HTTP:  www.nssl.noaa.gov/~lwicker
| Phone:    (405) 325-6340
| Fax:        (405) 325-6780
|
| "Programming is not just creating strings of instructions
| for a computer to execute.  It's also 'literary' in that you
| are trying to communicate a program structure to
| other humans reading the code." - Paul Rubin
|
|"Real efficiency comes from elegant solutions, not optimized programs.
| Optimization is always just a few correctness-preserving  
transformations
| away." - Jonathan Sobel
------------------------------------------------------------------------ 
----
|
| "The contents  of this message are mine personally and
| do not reflect any position of  the Government or NOAA."
|
------------------------------------------------------------------------ 
----

_______________________________________________
ncl-talk mailing list
ncl-talk_at_ucar.edu
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Tue Feb 06 2007 - 09:26:01 MST