Re: omega computation from CCSM3

Ola Carlos

This is strictly an issue of memory. I ran the original script. The
high-memory mark was
just over 3GB. (FYI: it took 4minutes) . Since your machine has "only"
2.5 GB of
total memory. It fails.

Why is the memory usage large?

[1] The input U and V variables are 442MB each; PS is 16MB .... total=900MB
              U and V: [time | 124] x [lev | 26] x [lat | 128] x [lon |
256]
              PS: [time | 124] x [lat | 128] x [lon | 256]

[2] The output OMEGA is also 442MB. So , (900MB + 442MB=1.342GB)

[3] You can look at NCL's omega_ccm_driver function located in
contributed.ncl via:

      less $NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl

     You will note that two spherical harmonic [SH] routines are used

          gradsg(psln,dpsl,dpsm) ; gradients of log(psfc)
gaussian grid
          div = uv2dvG(u,v) ; divergence on
gaussian grid

     The temporary "div" variable is another 442MB [1.392GB + 442MB=1.834GB]
                                                                                     

     The SH routines require internal temporary workspace.

[4] Note: even though the data are type "float"; All NCL computations
are done
     in double precision. Hence, memory is double in those instances. So the
     div = uv2dvG(u,v) the u and v variables would be promoted to
     double for computations. Then there is the workspace required
     by the SH routines.

The following script is slower due to use of the do loop . It took 5-6
minutes to finish
when N=4 and the maximum memory was 1.4GB. You could increase N [some
number which divides into ntim=124.

Note: the (/ ... /) surrounding the call to omega_ccm_driver suppresses
the transfer of meta data.

Good luck

load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"
load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"
load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"

begin
  ntim = 124

  fccm = addfile("b30.030e.cam2.h3.U.1960-01-01_cat_1960-01-31.nc", "r")
  fccm1 = addfile("b30.030e.cam2.h3.V.1960-01-01_cat_1960-01-31.nc", "r")
  fccm2 = addfile("b30.030e.cam2.h3.PS.1960-01-01_cat_1969-12-31.nc", "r")

  hyam = fccm->hyam
  hybm = fccm->hybm
  hyai = fccm->hyai
  hybi = fccm->hybi
                                                  ; PS(14600,128,256)
only read 124 times
  p0 = fccm2->P0
  psfc = fccm2->PS(0:ntim-1,:,:)
  u = fccm->U
  v = fccm1->V

  printVarSummary(psfc)
  printVarSummary( u )
  printVarSummary( v )

  omega = u ; same type/size/shape as u
                    ; all meta data copied

  N = 4
  M = N-1

  do nt=0,ntim-1,N
     omega(nt:nt+M,:,:,:) = (/ omega_ccm_driver(p0,psfc(nt:nt+M,:,:) \
                                          
,u(nt:nt+M,:,:,:),v(nt:nt+M,:,:,:) \
                                          ,hyam,hybm,hyai,hybi) /)
  end do

  omega_at_long_name = "Vertical pressure velocity"
  omega_at_units = "Pa/s"

  printVarSummary( omega)

end

Carlos Alberto Fernandes Marques wrote:
> Hi,
>
> I'm starting to use NCL. I need to compute the omega (Vertical
> pressure velocity) from the CCSM3 runs with the 6-hourly output stream
> (e.g."b30.030e"). When using NCL's omega_ccm_driver in the following
> script (to compute omega for January 1960):
>
> ========== begin script ==================================
> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"
> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"
> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"
>
> begin
>
> fccm = addfile("b30.030e.cam2.h3.U.1960-01-01_cat_1960-01-31.nc", "r")
> fccm1 = addfile("b30.030e.cam2.h3.V.1960-01-01_cat_1960-01-31.nc", "r")
> fccm2 = addfile("b30.030e.cam2.h3.PS.1960-01-01_cat_1969-12-31.nc",
> "r")
> hyam = fccm->hyam
> hybm = fccm->hybm
> hyai = fccm->hyai
> hybi = fccm->hybi
> p0 = fccm2->P0
> psfc = fccm2->PS(0:124-1,:,:)
> u = fccm->U
> v = fccm1->V
>
> omega = omega_ccm_driver(p0,psfc,u,v,hyam,hybm,hyai,hybi)
>
> end
> ========== end script ====================================
>
> I get the error messages:
> fatal:omega_ccm: Unable to allocate memory for output variable
> fatal:Execute: Error occurred at or near line 6989 in file
> $NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl
> fatal:Execute: Error occurred at or near line 18.
>
> So I sought the problem was on memory allocation. Then, I changed the
> script, putting the additional command (omega =
> new(dimsizes(u),typeof(u))) before the call of omega_ccm_driver. Then
> the error messages become:
> fatal:omega_ccm: Unable to coerce input variables to double precision
> fatal:Execute: Error occurred at or near line 6989 in file
> $NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl
> fatal:Execute: Error occurred at or near line 18.
>
>
> I have on my PC 2.5 GB of RAM, which I think it is enough. However, I
> tried the following script (on which omega is computed in two stages,
> i.e. for the first 80 times and then for the remaining 44 times), and
> it works:
>
> ========== begin script ==================================
> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"
> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"
> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"
>
> begin
>
> fccm = addfile("b30.030e.cam2.h3.U.1960-01-01_cat_1960-01-31.nc", "r")
> fccm1 = addfile("b30.030e.cam2.h3.V.1960-01-01_cat_1960-01-31.nc", "r")
> fccm2 = addfile("b30.030e.cam2.h3.PS.1960-01-01_cat_1969-12-31.nc",
> "r")
> hyam = fccm->hyam
> hybm = fccm->hybm
> hyai = fccm->hyai
> hybi = fccm->hybi
> p0 = fccm2->P0
> psfc = fccm2->PS(0:80-1,:,:)
> u = fccm->U(0:80-1,:,:,:)
> v = fccm1->V(0:80-1,:,:,:)
>
> omega1 = omega_ccm_driver(p0,psfc,u,v,hyam,hybm,hyai,hybi)
>
>
> psfc1 = fccm2->PS(80:124-1,:,:)
> u1 = fccm->U(80:124-1,:,:,:)
> v1 = fccm1->V(80:124-1,:,:,:)
>
> omega2 = omega_ccm_driver(p0,psfc1,u1,v1,hyam,hybm,hyai,hybi)
>
> omega = array_append_record(omega1,omega2,0)
>
> delete(omega1)
> delete(omega2)
>
> end
> ========== end script ====================================
>
>
> My questions are:
> - Is this way correct for computing omega from the CCSM3 runs?
> - If so, how to compute omega for a whole month?
>
> Thanks,
>
> Carlos.
>
> PS: Attached is the last script with additional commands to save the
> output omega to a netCDF file.
>
> =============================
> Physics Department,
> University of Aveiro.
> Campus de Santiago,
> 3810-193 Aveiro.
> Portugal
> =============================
> ------------------------------------------------------------------------
>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk_at_ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>

-- 
======================================================
Dennis J. Shea                  tel: 303-497-1361    |
P.O. Box 3000                   fax: 303-497-1333    |
Climate Analysis Section                             |
Climate & Global Dynamics Div.                       |
National Center for Atmospheric Research             |
Boulder, CO  80307                                   |
USA                        email: shea 'at' ucar.edu |
======================================================
_______________________________________________
ncl-talk mailing list
ncl-talk_at_ucar.edu
http://mailman.ucar.edu/mailman/listinfo/ncl-talk