Re: read OLR data

From: juki juki <juky_emc2_at_nyahnyahspammersnyahnyah>
Date: Mon Dec 12 2011 - 21:21:03 MST

OK, thank you very much..finally I take only certain years and in a given latitude and longitude, it works well. again, thank you ! Juky ________________________________ From: Dennis Shea <shea_at_ucar.edu> To: juki juki <juky_emc2_at_yahoo.com> Cc: "ncl-talk_at_ucar.edu" <ncl-talk_at_ucar.edu> Sent: Tuesday, December 13, 2011 1:07 PM Subject: Re: [ncl-talk] read OLR data [1] It has been called to my attention that the file is only     ~250MB ... not 2.5GB ... Sorry [2] If you are running out of *physical memory*         fatal:NclMalloc Failed:[errno=12]     Not much you can do if you want the full grid     and all the time steps.     If you want the (say) northern hemisphere   load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"   fileName = "olr.day.mean.nc <http://olr.day.mean.nc>"   inFile = addfile( fileName, "r" )   LATS = 0.0   LATN = 90.0   data = short2flt( inFile->olr(:,{LATS:LATN},:) )   printVarSummary(data)   printMinMax(data, 0) On 12/12/11 4:52 PM, juki juki wrote: > Dear Dr. Dennis; > > thanks for your reply. > >  >You do not have enough memory on your machine > > Don't you have any solution for that ? Currently I am working in Cygwin. > > Best regards, > Marzuki > > > ------------------------------------------------------------------------ > *From:* Dennis Shea <shea_at_ucar.edu> > *To:* juki juki <juky_emc2_at_yahoo.com> > *Cc:* "ncl-talk_at_ucar.edu" <ncl-talk_at_ucar.edu> > *Sent:* Tuesday, December 13, 2011 8:36 AM > *Subject:* Re: [ncl-talk] read OLR data > > fatal:NclMalloc Failed:[errno=12] > > You do not have enough memory on your machine. > >  > Variable: dataShort >  > Type: short >  > Total Size: 249870240 bytes > > The variable is ~2.5GB in *short* form. > > In float form, it will be ~5GB. > > For the conversion to occur, both the short and float > must be in memory at the same time. > > > ==== > > The following reads only 10 time steps but it shows the > steps you used are correct. > > load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl" > > fileName = "olr.day.mean.nc <http://olr.day.mean.nc>" > inFile = addfile( fileName, "r" ) > itStrt = 0 > itLast = 9 > > data = short2flt( inFile->olr(itStrt:itLast,:,:) ) > > printVarSummary(data) > printMinMax(data, 0) > > > > > On 12/12/11 4:14 PM, juki juki wrote: >  > Dear Dr. Dennis; >  > >  > Thanks for your reply. The following is detailed error comment: >  > >  > Copyright (C) 1995-2011 - All Rights Reserved >  > University Corporation for Atmospheric Research >  > NCAR Command Language Version 6.0.0 >  > The use of this software is governed by a License Agreement. >  > See http://www.ncl.ucar.edu/ for more details. >  > (0) Here we go! Tue Dec 13 08:06:35 TST 2011 >  > (0) Reading the data... >  > fatal:NclMalloc Failed:[errno=12] >  > >  > fatal:Could not coerce values for operation >  > >  > fatal:Execute: Error occurred at or near line 1696 in file >  > $NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl >  > >  > >  > fatal:Execute: Error occurred at or near line 33 in file test1.ncl >  > >  > >You should send the output from print(dataShort) >  > I can not send it completely here because dataShort are from >  > "olr.day.mean.nc <http://olr.day.mean.nc/>", it is very big file, but >  > the beginning part of the output is >  > >  > Variable: dataShort >  > Type: short >  > Total Size: 249870240 bytes >  > 124935120 values >  > Number of Dimensions: 3 >  > Dimensions and sizes: [time | 11885] x [lat | 73] x [lon | 144] >  > Coordinates: >  > time: [17338824..17624040] >  > lat: [90..-90] >  > lon: [ 0..357.5] >  > Number Of Attributes: 15 >  > long_name : Daily OLR >  > unpacked_valid_range : ( 0, 500 ) >  > actual_range : ( 94.5, 328.25 ) >  > units : W/m^2 >  > add_offset : 327.65 >  > scale_factor : 0.01 >  > missing_value : 32766 >  > var_desc : Outgoing Longwave Radiation >  > precision : 2 >  > dataset : NOAA Interpolated OLR >  > level_desc : Other >  > statistic : Mean >  > parent_stat : Individual Obs >  > valid_range : ( -32765, 17235 ) >  > _FillValue : 32766 >  > (0,0,0) -17090 >  > (0,0,1) -17090 >  > (0,0,2) -17090 >  > (0,0,3) -17090 >  > (0,0,4) -17090 >  > (0,0,5) -17090 >  > (0,0,6) -17090 >  > (0,0,7) -17090 >  > (0,0,8) -17090 >  > (0,0,9) -17090 >  > ........ >  > >  > Thanks again for best help. >  > >  > Regards, >  > Juky >  > >  > >  > >  > >  > >  > > inFile = addfile( fileName, "r" ) >  > > obsPerDay = 1 >  > > >  > > >  > > ; find our times >  > > time = inFile->time >  > > filtStart = ut_inv_calendar( 1979, 01, 01, 0, 0, 0, time_at_units, 0) >  > > filtEnd = max(time) >  > > startDate = filtStart + 192 * 24 >  > > endDate = filtEnd - 192 * 24 >  > > >  > > ; read in the unfiltered data >  > > print( "Reading the data..." ) >  > > dataShort = inFile->olr({filtStart:filtEnd},:,:) >  > >  > >  > ------------------------------------------------------------------------ >  > *From:* Dennis Shea <shea_at_ucar.edu <mailto:shea_at_ucar.edu>> >  > *To:* juki juki <juky_emc2_at_yahoo.com <mailto:juky_emc2_at_yahoo.com>> >  > *Cc:* "ncl-talk_at_ucar.edu <mailto:ncl-talk_at_ucar.edu>" > <ncl-talk_at_ucar.edu <mailto:ncl-talk_at_ucar.edu>> >  > *Sent:* Monday, December 12, 2011 11:40 PM >  > *Subject:* Re: [ncl-talk] convert nc file in sigma to pressure level >  > >  > There is not enough information. >  > Not sure what "[errno=12] is. >  > >  > > ;print(dataShort) >  > > data = short2flt( dataShort ) >  > >  > You should send the output from >  > >  > print(dataShort) >  > >  > On 12/11/11 10:45 PM, juki juki wrote: >  > > Hi Dr. Dennis; >  > > >  > > I want to read the OLR data with the following code: >  > > >  > > load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl" >  > > load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl" >  > > load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl" >  > > >  > > >  > > begin >  > > >  > > print( "Here we go! " + systemfunc( "date" ) ) >  > > >  > > mis = -999 >  > > mis@_FillValue = -999 >  > > >  > > ; open the file >  > > >  > > fileName = "olr.day.mean.nc <http://olr.day.mean.nc>" >  > > inFile = addfile( fileName, "r" ) >  > > obsPerDay = 1 >  > > >  > > >  > > ; find our times >  > > time = inFile->time >  > > filtStart = ut_inv_calendar( 1979, 01, 01, 0, 0, 0, time_at_units, 0) >  > > filtEnd = max(time) >  > > startDate = filtStart + 192 * 24 >  > > endDate = filtEnd - 192 * 24 >  > > >  > > ; read in the unfiltered data >  > > print( "Reading the data..." ) >  > > dataShort = inFile->olr({filtStart:filtEnd},:,:) >  > > ;print(dataShort) >  > > data = short2flt( dataShort ) >  > > >  > > ; ......many more >  > > end >  > > >  > > the code stop at data = short2flt( dataShort ), with the [errno=12]. I >  > > am working in cygwin, could you suggest some help ? >  > > >  > > >  > > thanks very much >  > > Juky >  > > >  > > >  > > >  > > >  > > > ------------------------------------------------------------------------ >  > > *From:* Dennis Shea <shea_at_ucar.edu <mailto:shea_at_ucar.edu> > <mailto:shea_at_ucar.edu <mailto:shea_at_ucar.edu>>> >  > > *To:* Coly SAR <colysar_at_gmail.com <mailto:colysar_at_gmail.com> > <mailto:colysar_at_gmail.com <mailto:colysar_at_gmail.com>>> >  > > *Cc:* ncl-talk_at_ucar.edu <mailto:ncl-talk_at_ucar.edu> > <mailto:ncl-talk_at_ucar.edu <mailto:ncl-talk_at_ucar.edu>> >  > > *Sent:* Sunday, September 4, 2011 11:21 PM >  > > *Subject:* Re: [ncl-talk] convert nc file in sigma to pressure level >  > > >  > > Not sure what to say. See Example 2 at >  > > http://www.ncl.ucar.edu/Applications/vert_interp.shtml >  > > or Example 5 at >  > > http://www.ncl.ucar.edu/Document/Functions/Built-in/vinth2p.shtml >  > > >  > > =========================================== >  > > You *MUST* have the surface pressure. >  > > >  > > Here is an excerpt: >  > > >  > > f = addfile(....) >  > > t = f->t >  > > ps= f->ps ; sfc pressure [hPa or Pa] >  > > >  > > sigma = f->sigma >  > > or >  > > sigma = t&lev >  > > >  > > dimt = dimsizes(t) >  > > ntim = dimt(0) >  > > ksig = dimt(1) >  > > nlat = dimt(2) >  > > mlon = dimt(3) >  > > >  > > ;*************************************************** >  > > ; interpolate to pressure levels >  > > ;*************************************************** >  > > plevs = (/1000,925, 850, 800, 700, 600, 500, 300 \ >  > > , 200, 250, 200, 150, 100/)*1.0 >  > > plevs_at_units = "hPa" >  > > >  > > if (ps_at_units.eq <mailto:ps_at_units.eq> <mailto:ps_at_units.eq > <mailto:ps_at_units.eq>> <mailto:ps_at_units.eq <mailto:ps_at_units.eq> >  > <mailto:ps_at_units.eq <mailto:ps_at_units.eq>>>."Pa" .or. ps_at_units.eq > <mailto:ps_at_units.eq> <mailto:ps_at_units.eq <mailto:ps_at_units.eq>> >  > > <mailto:ps_at_units.eq <mailto:ps_at_units.eq> <mailto:ps_at_units.eq > <mailto:ps_at_units.eq>>>."Pascals") then >  > > ps = ps*0.01 >  > > ps_at_units = "hPa" >  > > end if >  > > >  > > hyam = fspan(0.,0.,ksig-1) ; create fake hyba array >  > > hybm = sigma >  > > >  > > tp = vinth2p(t,hyam,hybm,plevs,ps,1,1000,1,True) >  > > printVarSummary(tp) >  > > >  > > >  > > On 9/4/11 5:00 AM, Coly SAR wrote: >  > > > Below is the structure of the file i am trying to convert in pressure >  > > > levels. It 18 sigma levels and I want to have at standard pressure >  > > > levels : 1000 925 850 800 700 600 500 300 200 250 200 150 100 >  > > > >  > > > CS >  > > > >  > > > netcdf atm99 { >  > > > dimensions: >  > > > lon = 190 ; >  > > > lat = 106 ; >  > > > lev = 18 ; >  > > > time = UNLIMITED ; // (124 currently) >  > > > variables: >  > > > double lon(lon) ; >  > > > lon:long_name = "longitude" ; >  > > > lon:units = "degrees_east" ; >  > > > lon:standard_name = "longitude" ; >  > > > lon:axis = "X" ; >  > > > double lat(lat) ; >  > > > lat:long_name = "latitude" ; >  > > > lat:units = "degrees_north" ; >  > > > lat:standard_name = "latitude" ; >  > > > lat:axis = "Y" ; >  > > > double lev(lev) ; >  > > > lev:long_name = "generic" ; >  > > > lev:units = "level" ; >  > > > lev:axis = "Z" ; >  > > > double time(time) ; >  > > > time:units = "hours since 1999-01-01 06:00:00" ; >  > > > time:calendar = "standard" ; >  > > > float u(time, lev, lat, lon) ; >  > > > u:long_name = "westerly wind (m/s)" ; >  > > > u:_FillValue = -1.e+34f ; >  > > > float v(time, lev, lat, lon) ; >  > > > v:long_name = "southerly wind (m/s)" ; >  > > > v:_FillValue = -1.e+34f ; >  > > > float w(time, lev, lat, lon) ; >  > > > w:long_name = "omega (hPa/s) p-velocity" ; >  > > > w:_FillValue = -1.e+34f ; >  > > > float t(time, lev, lat, lon) ; >  > > > t:long_name = "air temperature (degree)" ; >  > > > t:_FillValue = -1.e+34f ; >  > > > >  > > > >  > > > 2011/9/3 Coly SAR <colysar_at_gmail.com <mailto:colysar_at_gmail.com> > <mailto:colysar_at_gmail.com <mailto:colysar_at_gmail.com>> >  > <mailto:colysar_at_gmail.com <mailto:colysar_at_gmail.com> > <mailto:colysar_at_gmail.com <mailto:colysar_at_gmail.com>>> >  > > <mailto:colysar_at_gmail.com <mailto:colysar_at_gmail.com> > <mailto:colysar_at_gmail.com <mailto:colysar_at_gmail.com>> >  > <mailto:colysar_at_gmail.com <mailto:colysar_at_gmail.com> > <mailto:colysar_at_gmail.com <mailto:colysar_at_gmail.com>>>>> >  > > > >  > > > >  > > > It is essentially interpolating sigma levels at selected constant >  > > > pressure >  > > > levels and write it to a new nc file having pressure levels. >  > > > >  > > > CS >  > > > >  > > > >  > > > 2011/9/3 Dennis Shea <shea_at_ucar.edu <mailto:shea_at_ucar.edu> > <mailto:shea_at_ucar.edu <mailto:shea_at_ucar.edu>> >  > <mailto:shea_at_ucar.edu <mailto:shea_at_ucar.edu> <mailto:shea_at_ucar.edu > <mailto:shea_at_ucar.edu>>> >  > > <mailto:shea_at_ucar.edu <mailto:shea_at_ucar.edu> <mailto:shea_at_ucar.edu > <mailto:shea_at_ucar.edu>> <mailto:shea_at_ucar.edu <mailto:shea_at_ucar.edu> >  > <mailto:shea_at_ucar.edu <mailto:shea_at_ucar.edu>>>>> >  > > > >  > > > This questions is a bit ambiguous. Are you talking about calculating >  > > > the pressure at each level and grid point or interpolating >  > > > sigm to constant pressure levels? >  > > > >  > > > See Example 2 at: >  > > > http://www.ncl.ucar.edu/__Applications/vert_interp.shtml >  > > > <http://www.ncl.ucar.edu/Applications/vert_interp.shtml> >  > > > >  > > > It shows how to do both. >  > > > >  > > > >  > > > 9/3/11 7:37 AM, Coly SAR wrote: >  > > > >  > > > Dear NCL users, >  > > > I have model output in netcdf with sigma levels and want to >  > > > convert >  > > > from sigma to pressure and wondering if a such >  > > > routine or script is available in ncl. >  > > > >  > > > CS >  > > > >  > > > >  > > > _________________________________________________ >  > > > ncl-talk mailing list >  > > > List instructions, subscriber options, unsubscribe: >  > > > http://mailman.ucar.edu/__mailman/listinfo/ncl-talk >  > > > <http://mailman.ucar.edu/mailman/listinfo/ncl-talk> >  > > > >  > > > >  > > > >  > > > >  > > > >  > > > _______________________________________________ >  > > > ncl-talk mailing list >  > > > List instructions, subscriber options, unsubscribe: >  > > > http://mailman.ucar.edu/mailman/listinfo/ncl-talk >  > > _______________________________________________ >  > > ncl-talk mailing list >  > > List instructions, subscriber options, unsubscribe: >  > > http://mailman.ucar.edu/mailman/listinfo/ncl-talk >  > > >  > > >  > >  > > >

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Mon Dec 12 21:21:16 2011

This archive was generated by hypermail 2.1.8 : Fri Dec 16 2011 - 10:27:49 MST