Re: read OLR data

From: Dennis Shea <shea_at_nyahnyahspammersnyahnyah>
Date: Mon Dec 12 2011 - 21:07:37 MST

[1] It has been called to my attention that the file is only
     ~250MB ... not 2.5GB ... Sorry

[2] If you are running out of *physical memory*
        fatal:NclMalloc Failed:[errno=12]
     Not much you can do if you want the full grid
     and all the time steps.

     If you want the (say) northern hemisphere

  load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"

  fileName = "olr.day.mean.nc <http://olr.day.mean.nc>"
  inFile = addfile( fileName, "r" )

  LATS = 0.0
  LATN = 90.0
  data = short2flt( inFile->olr(:,{LATS:LATN},:) )

  printVarSummary(data)
  printMinMax(data, 0)

On 12/12/11 4:52 PM, juki juki wrote:
> Dear Dr. Dennis;
>
> thanks for your reply.
>
> >You do not have enough memory on your machine
>
> Don't you have any solution for that ? Currently I am working in Cygwin.
>
> Best regards,
> Marzuki
>
>
> ------------------------------------------------------------------------
> *From:* Dennis Shea <shea@ucar.edu>
> *To:* juki juki <juky_emc2@yahoo.com>
> *Cc:* "ncl-talk@ucar.edu" <ncl-talk@ucar.edu>
> *Sent:* Tuesday, December 13, 2011 8:36 AM
> *Subject:* Re: [ncl-talk] read OLR data
>
> fatal:NclMalloc Failed:[errno=12]
>
> You do not have enough memory on your machine.
>
> > Variable: dataShort
> > Type: short
> > Total Size: 249870240 bytes
>
> The variable is ~2.5GB in *short* form.
>
> In float form, it will be ~5GB.
>
> For the conversion to occur, both the short and float
> must be in memory at the same time.
>
>
> ====
>
> The following reads only 10 time steps but it shows the
> steps you used are correct.
>
> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"
>
> fileName = "olr.day.mean.nc <http://olr.day.mean.nc>"
> inFile = addfile( fileName, "r" )
> itStrt = 0
> itLast = 9
>
> data = short2flt( inFile->olr(itStrt:itLast,:,:) )
>
> printVarSummary(data)
> printMinMax(data, 0)
>
>
>
>
> On 12/12/11 4:14 PM, juki juki wrote:
> > Dear Dr. Dennis;
> >
> > Thanks for your reply. The following is detailed error comment:
> >
> > Copyright (C) 1995-2011 - All Rights Reserved
> > University Corporation for Atmospheric Research
> > NCAR Command Language Version 6.0.0
> > The use of this software is governed by a License Agreement.
> > See http://www.ncl.ucar.edu/ for more details.
> > (0) Here we go! Tue Dec 13 08:06:35 TST 2011
> > (0) Reading the data...
> > fatal:NclMalloc Failed:[errno=12]
> >
> > fatal:Could not coerce values for operation
> >
> > fatal:Execute: Error occurred at or near line 1696 in file
> > $NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl
> >
> >
> > fatal:Execute: Error occurred at or near line 33 in file test1.ncl
> >
> > >You should send the output from print(dataShort)
> > I can not send it completely here because dataShort are from
> > "olr.day.mean.nc <http://olr.day.mean.nc/>", it is very big file, but
> > the beginning part of the output is
> >
> > Variable: dataShort
> > Type: short
> > Total Size: 249870240 bytes
> > 124935120 values
> > Number of Dimensions: 3
> > Dimensions and sizes: [time | 11885] x [lat | 73] x [lon | 144]
> > Coordinates:
> > time: [17338824..17624040]
> > lat: [90..-90]
> > lon: [ 0..357.5]
> > Number Of Attributes: 15
> > long_name : Daily OLR
> > unpacked_valid_range : ( 0, 500 )
> > actual_range : ( 94.5, 328.25 )
> > units : W/m^2
> > add_offset : 327.65
> > scale_factor : 0.01
> > missing_value : 32766
> > var_desc : Outgoing Longwave Radiation
> > precision : 2
> > dataset : NOAA Interpolated OLR
> > level_desc : Other
> > statistic : Mean
> > parent_stat : Individual Obs
> > valid_range : ( -32765, 17235 )
> > _FillValue : 32766
> > (0,0,0) -17090
> > (0,0,1) -17090
> > (0,0,2) -17090
> > (0,0,3) -17090
> > (0,0,4) -17090
> > (0,0,5) -17090
> > (0,0,6) -17090
> > (0,0,7) -17090
> > (0,0,8) -17090
> > (0,0,9) -17090
> > ........
> >
> > Thanks again for best help.
> >
> > Regards,
> > Juky
> >
> >
> >
> >
> >
> > > inFile = addfile( fileName, "r" )
> > > obsPerDay = 1
> > >
> > >
> > > ; find our times
> > > time = inFile->time
> > > filtStart = ut_inv_calendar( 1979, 01, 01, 0, 0, 0, time@units, 0)
> > > filtEnd = max(time)
> > > startDate = filtStart + 192 * 24
> > > endDate = filtEnd - 192 * 24
> > >
> > > ; read in the unfiltered data
> > > print( "Reading the data..." )
> > > dataShort = inFile->olr({filtStart:filtEnd},:,:)
> >
> >
> > ------------------------------------------------------------------------
> > *From:* Dennis Shea <shea@ucar.edu <mailto:shea@ucar.edu>>
> > *To:* juki juki <juky_emc2@yahoo.com <mailto:juky_emc2@yahoo.com>>
> > *Cc:* "ncl-talk@ucar.edu <mailto:ncl-talk@ucar.edu>"
> <ncl-talk@ucar.edu <mailto:ncl-talk@ucar.edu>>
> > *Sent:* Monday, December 12, 2011 11:40 PM
> > *Subject:* Re: [ncl-talk] convert nc file in sigma to pressure level
> >
> > There is not enough information.
> > Not sure what "[errno=12] is.
> >
> > > ;print(dataShort)
> > > data = short2flt( dataShort )
> >
> > You should send the output from
> >
> > print(dataShort)
> >
> > On 12/11/11 10:45 PM, juki juki wrote:
> > > Hi Dr. Dennis;
> > >
> > > I want to read the OLR data with the following code:
> > >
> > > load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"
> > > load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"
> > > load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"
> > >
> > >
> > > begin
> > >
> > > print( "Here we go! " + systemfunc( "date" ) )
> > >
> > > mis = -999
> > > mis@_FillValue = -999
> > >
> > > ; open the file
> > >
> > > fileName = "olr.day.mean.nc <http://olr.day.mean.nc>"
> > > inFile = addfile( fileName, "r" )
> > > obsPerDay = 1
> > >
> > >
> > > ; find our times
> > > time = inFile->time
> > > filtStart = ut_inv_calendar( 1979, 01, 01, 0, 0, 0, time@units, 0)
> > > filtEnd = max(time)
> > > startDate = filtStart + 192 * 24
> > > endDate = filtEnd - 192 * 24
> > >
> > > ; read in the unfiltered data
> > > print( "Reading the data..." )
> > > dataShort = inFile->olr({filtStart:filtEnd},:,:)
> > > ;print(dataShort)
> > > data = short2flt( dataShort )
> > >
> > > ; ......many more
> > > end
> > >
> > > the code stop at data = short2flt( dataShort ), with the [errno=12]. I
> > > am working in cygwin, could you suggest some help ?
> > >
> > >
> > > thanks very much
> > > Juky
> > >
> > >
> > >
> > >
> > >
> ------------------------------------------------------------------------
> > > *From:* Dennis Shea <shea@ucar.edu <mailto:shea@ucar.edu>
> <mailto:shea@ucar.edu <mailto:shea@ucar.edu>>>
> > > *To:* Coly SAR <colysar@gmail.com <mailto:colysar@gmail.com>
> <mailto:colysar@gmail.com <mailto:colysar@gmail.com>>>
> > > *Cc:* ncl-talk@ucar.edu <mailto:ncl-talk@ucar.edu>
> <mailto:ncl-talk@ucar.edu <mailto:ncl-talk@ucar.edu>>
> > > *Sent:* Sunday, September 4, 2011 11:21 PM
> > > *Subject:* Re: [ncl-talk] convert nc file in sigma to pressure level
> > >
> > > Not sure what to say. See Example 2 at
> > > http://www.ncl.ucar.edu/Applications/vert_interp.shtml
> > > or Example 5 at
> > > http://www.ncl.ucar.edu/Document/Functions/Built-in/vinth2p.shtml
> > >
> > > ===========================================
> > > You *MUST* have the surface pressure.
> > >
> > > Here is an excerpt:
> > >
> > > f = addfile(....)
> > > t = f->t
> > > ps= f->ps ; sfc pressure [hPa or Pa]
> > >
> > > sigma = f->sigma
> > > or
> > > sigma = t&lev
> > >
> > > dimt = dimsizes(t)
> > > ntim = dimt(0)
> > > ksig = dimt(1)
> > > nlat = dimt(2)
> > > mlon = dimt(3)
> > >
> > > ;***************************************************
> > > ; interpolate to pressure levels
> > > ;***************************************************
> > > plevs = (/1000,925, 850, 800, 700, 600, 500, 300 \
> > > , 200, 250, 200, 150, 100/)*1.0
> > > plevs@units = "hPa"
> > >
> > > if (ps@units.eq <mailto:ps@units.eq> <mailto:ps@units.eq
> <mailto:ps@units.eq>> <mailto:ps@units.eq <mailto:ps@units.eq>
> > <mailto:ps@units.eq <mailto:ps@units.eq>>>."Pa" .or. ps@units.eq
> <mailto:ps@units.eq> <mailto:ps@units.eq <mailto:ps@units.eq>>
> > > <mailto:ps@units.eq <mailto:ps@units.eq> <mailto:ps@units.eq
> <mailto:ps@units.eq>>>."Pascals") then
> > > ps = ps*0.01
> > > ps@units = "hPa"
> > > end if
> > >
> > > hyam = fspan(0.,0.,ksig-1) ; create fake hyba array
> > > hybm = sigma
> > >
> > > tp = vinth2p(t,hyam,hybm,plevs,ps,1,1000,1,True)
> > > printVarSummary(tp)
> > >
> > >
> > > On 9/4/11 5:00 AM, Coly SAR wrote:
> > > > Below is the structure of the file i am trying to convert in pressure
> > > > levels. It 18 sigma levels and I want to have at standard pressure
> > > > levels : 1000 925 850 800 700 600 500 300 200 250 200 150 100
> > > >
> > > > CS
> > > >
> > > > netcdf atm99 {
> > > > dimensions:
> > > > lon = 190 ;
> > > > lat = 106 ;
> > > > lev = 18 ;
> > > > time = UNLIMITED ; // (124 currently)
> > > > variables:
> > > > double lon(lon) ;
> > > > lon:long_name = "longitude" ;
> > > > lon:units = "degrees_east" ;
> > > > lon:standard_name = "longitude" ;
> > > > lon:axis = "X" ;
> > > > double lat(lat) ;
> > > > lat:long_name = "latitude" ;
> > > > lat:units = "degrees_north" ;
> > > > lat:standard_name = "latitude" ;
> > > > lat:axis = "Y" ;
> > > > double lev(lev) ;
> > > > lev:long_name = "generic" ;
> > > > lev:units = "level" ;
> > > > lev:axis = "Z" ;
> > > > double time(time) ;
> > > > time:units = "hours since 1999-01-01 06:00:00" ;
> > > > time:calendar = "standard" ;
> > > > float u(time, lev, lat, lon) ;
> > > > u:long_name = "westerly wind (m/s)" ;
> > > > u:_FillValue = -1.e+34f ;
> > > > float v(time, lev, lat, lon) ;
> > > > v:long_name = "southerly wind (m/s)" ;
> > > > v:_FillValue = -1.e+34f ;
> > > > float w(time, lev, lat, lon) ;
> > > > w:long_name = "omega (hPa/s) p-velocity" ;
> > > > w:_FillValue = -1.e+34f ;
> > > > float t(time, lev, lat, lon) ;
> > > > t:long_name = "air temperature (degree)" ;
> > > > t:_FillValue = -1.e+34f ;
> > > >
> > > >
> > > > 2011/9/3 Coly SAR <colysar@gmail.com <mailto:colysar@gmail.com>
> <mailto:colysar@gmail.com <mailto:colysar@gmail.com>>
> > <mailto:colysar@gmail.com <mailto:colysar@gmail.com>
> <mailto:colysar@gmail.com <mailto:colysar@gmail.com>>>
> > > <mailto:colysar@gmail.com <mailto:colysar@gmail.com>
> <mailto:colysar@gmail.com <mailto:colysar@gmail.com>>
> > <mailto:colysar@gmail.com <mailto:colysar@gmail.com>
> <mailto:colysar@gmail.com <mailto:colysar@gmail.com>>>>>
> > > >
> > > >
> > > > It is essentially interpolating sigma levels at selected constant
> > > > pressure
> > > > levels and write it to a new nc file having pressure levels.
> > > >
> > > > CS
> > > >
> > > >
> > > > 2011/9/3 Dennis Shea <shea@ucar.edu <mailto:shea@ucar.edu>
> <mailto:shea@ucar.edu <mailto:shea@ucar.edu>>
> > <mailto:shea@ucar.edu <mailto:shea@ucar.edu> <mailto:shea@ucar.edu
> <mailto:shea@ucar.edu>>>
> > > <mailto:shea@ucar.edu <mailto:shea@ucar.edu> <mailto:shea@ucar.edu
> <mailto:shea@ucar.edu>> <mailto:shea@ucar.edu <mailto:shea@ucar.edu>
> > <mailto:shea@ucar.edu <mailto:shea@ucar.edu>>>>>
> > > >
> > > > This questions is a bit ambiguous. Are you talking about calculating
> > > > the pressure at each level and grid point or interpolating
> > > > sigm to constant pressure levels?
> > > >
> > > > See Example 2 at:
> > > > http://www.ncl.ucar.edu/__Applications/vert_interp.shtml
> > > > <http://www.ncl.ucar.edu/Applications/vert_interp.shtml>
> > > >
> > > > It shows how to do both.
> > > >
> > > >
> > > > 9/3/11 7:37 AM, Coly SAR wrote:
> > > >
> > > > Dear NCL users,
> > > > I have model output in netcdf with sigma levels and want to
> > > > convert
> > > > from sigma to pressure and wondering if a such
> > > > routine or script is available in ncl.
> > > >
> > > > CS
> > > >
> > > >
> > > > _________________________________________________
> > > > ncl-talk mailing list
> > > > List instructions, subscriber options, unsubscribe:
> > > > http://mailman.ucar.edu/__mailman/listinfo/ncl-talk
> > > > <http://mailman.ucar.edu/mailman/listinfo/ncl-talk>
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > ncl-talk mailing list
> > > > List instructions, subscriber options, unsubscribe:
> > > > http://mailman.ucar.edu/mailman/listinfo/ncl-talk
> > > _______________________________________________
> > > ncl-talk mailing list
> > > List instructions, subscriber options, unsubscribe:
> > > http://mailman.ucar.edu/mailman/listinfo/ncl-talk
> > >
> > >
> >
> >
>
>
_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Mon Dec 12 21:07:47 2011

This archive was generated by hypermail 2.1.8 : Fri Dec 16 2011 - 10:27:49 MST