Re: ncl-talk Digest, Vol 124, Issue 38

I love you xoxoxoxoxoxoxoxoxooxxo

03/27/2014 02:00 PM, sgrìobh ncl-talk-request@ucar.edu:
> Send ncl-talk mailing list submissions to
> ncl-talk@ucar.edu
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
> or, via email, send a message with subject or body 'help' to
> ncl-talk-request@ucar.edu
>
> You can reach the person managing the list at
> ncl-talk-owner@ucar.edu
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of ncl-talk digest..."
>
>
> Today's Topics:
>
> 1. Re: Fatal Error Message (Dennis Shea)
> 2. Re: Vertical Integration of WRF PM10 with Known Thickness
> Lengths; Problems with Integration and Coordination of Dimensions
> (Dennis Shea)
> 3. Re: Fatal Error Message (Lisa Alexander)
> 4. More than 2 Dimensions for "fft2df " (Ghassan J. Alaka, Jr.)
> 5. Re: How to put user specified variable name in
> "wrf_PressureLevel3.ncl" example plot (Mary Haley)
> 6. Re: Polar stereographic map bug! (Mary Haley)
> 7. Re: Plot precipitation with multiple WRF files (Mary Haley)
> 8. Re: How to put user specified variable name in
> "wrf_PressureLevel3.ncl" example plot (mmkamal@uwaterloo.ca)
> 9. Re: Vertical Integration of WRF PM10 with Known Thickness
> Lengths; Problems with Integration and Coordination of Dimensions
> (James Madden)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Thu, 27 Mar 2014 07:20:43 -0600
> From: Dennis Shea <shea@ucar.edu>
> Subject: Re: Fatal Error Message
> To: Lisa Alexander <lisa_s_a@yahoo.ca>
> Cc: "ncl-talk@ucar.edu" <ncl-talk@ucar.edu>
> Message-ID: <533425AB.3010300@ucar.edu>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
>
> [1] fatal:NclMalloc Failed:[errno=12]
>
> "I have been running a program on my computer, but this time
> the case time length is very long ..."
>
> I assume this means the array size(s) are much larger.
>
> You have run out of memory.
>
> If the primary variable is (say) 6GB and you have only 4GB,
> you will get a 'NclMalloc Failed' error. This is regardless
> if you have deleted all other variables.
>
> Do you know how much memory your computer has *available*?
> Do you know how much memory your main variable(s) need?
>
> [2] One crude approach
>
> %> ncl lisaAlexander.ncl >&! z.out &
> Then immediately,
> %> top
>
> This will show you the memory usage of your job.
>
>
> On 3/26/14, 4:12 PM, Lisa Alexander wrote:
>> Hello,
>> Please see the problem I have been experiencing below (noted in a previous post). I have tried to delete all variables once they are not being used and this does not work (does not improve the situation even slightly).
>> Would it be possible for someone to take a look at the program to see if there are any obvious sinks for computer memory?
>> Thanks for your help,
>> Lisa
>>
>> --- On Wed, 3/26/14, Lisa Alexander <lisa_s_a@yahoo.ca> wrote:
>>
>>> From: Lisa Alexander <lisa_s_a@yahoo.ca>
>>> Subject: Fatal Error Message
>>> To: ncl-talk@ucar.edu
>>> Received: Wednesday, March 26, 2014, 2:56 PM
>>> Hello,
>>> I have been running a program on my computer, but this time
>>> the case time length is very long. When I run it, it stops
>>> part way through and gives me this message:
>>>
>>> fatal:NclMalloc Failed:[errno=12]
>>> Segmentation fault (core dumped)
>>>
>>> I must say, whenever I run the program (this time and other
>>> times), the program runs very quickly to begin with, but
>>> slows down over time.
>>>
>>> Can somebody let me know what the error message means?
>>> Thanks,
>>> Lisa
>>> _______________________________________________
>>> ncl-talk mailing list
>>> List instructions, subscriber options, unsubscribe:
>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>
>> _______________________________________________
>> ncl-talk mailing list
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>
>
> ------------------------------
>
> Message: 2
> Date: Thu, 27 Mar 2014 07:37:19 -0600
> From: Dennis Shea <shea@ucar.edu>
> Subject: Re: Vertical Integration of WRF PM10 with Known
> Thickness Lengths; Problems with Integration and Coordination of
> Dimensions
> To: James Madden <jmmadden@alaska.edu>, ncl-talk@ucar.edu
> Message-ID: <5334298F.8010405@ucar.edu>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> [1] Eliminate the entire 'do it=' loop
> [2] You do not need: thick = new((/27/),float) ; define a new array
> NCL can determine the size of 'thick automatically'
> [3] Read documentation for conform_dims (conform, also), dim_sum_n
> and getfilevardimsizes
> [4] It is recommended to use the dim_*_n over the older dim_* functions.
> The latter are kept for backward compatibility only.
>
> ;--------
> a = addfile(...,"r")
> dimpm = getfilevardimsizes(a,"PM10") ; (0,1,2,3)
> thick = (/ ... /) ; (klev) => (27)
> THICK = conform_dims(dimpm, thick, 1) ; dim #1 is klev
> printVarSummary(THICK) ; (ntim,klev,nlat,mlon)
>
> pm = a->PM10 ; (ntim,klev,nlat,mlon)
> printVarSummary(pm)
>
> vertint = dim_sum_n(pm*THICK,1) ; (ntim,nlat,mlon)
> printVarSummary(vertint)
> delete(THICK) ; not necessary
>
> copy_VarCoords(pm(:,0,:,:), vertint) ; copy appropriate coord info
> vertint@long_name = "..." ; assign attributes
> vertint@units = "..."
> printVarSummary(vertint)
> ;--------
>
> On 3/26/14, 4:45 PM, James Madden wrote:
>> Hello,
>>
>> I am trying to perform a vertical integration of PM10 in layers across the
>> entire domain. I already have the thicknesses of each layer.
>>
>> However, I am unable to perform the integration, and coordinate the arrays
>> into something manageable for plotting.
>>
>> I have attached the code below. In the code, I have notated where I have
>> stumbled into problems, and I give explanations at those spots.
>>
>> Am I missing something much simpler? I feel like I am making this problem
>> way too complex.
>>
>> I have a dimension issue, but I have handled these problems before, and if
>> I introduce more loops to handle the latitude and longitude grid cells, my
>> plots will take way too long to form.
>>
>> The problem is later in the code, after all of the resources.
>>
>> Thank you much,
>>
>> Mike
>>
>> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"
>> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"
>> load "$NCARG_ROOT/lib/ncarg/nclscripts/wrf/WRF_contributed.ncl"
>> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/shea_util.ncl"
>> load "$NCARG_ROOT/lib/ncarg/nclscripts/wrf/WRFUserARW.ncl"
>>
>> begin
>>
>> wks = gsn_open_wks("x11","vert_int_plot")
>> gsn_define_colormap(wks,"BlAqGrYeOrReVi200")
>>
>> monthname = "June"
>> month = "06"
>> day = "24"
>> year = "2009"
>>
>> ; Read WRF File
>> a =
>> addfile("/import/c/w/jmmadden/WRFruns/wrfout_d01_2009-"+month+"-"+day+"_00:00:00"+".nc","r")
>>
>> ; Set Resources
>> ;----------------------------------------------------------------------
>> ; Begin plotting
>> ;----------------------------------------------------------------------
>> res = True ; plot mods desired
>> res@gsnMaximize = True ; uncomment to maximize size
>> res@gsnSpreadColors = True ; use full range of colormap
>> res@cnFillOn = True ; color plot desired
>> res@cnLinesOn = False ; turn off contour lines
>> res@cnLineLabelsOn = False ; turn off contour labels
>> ; res@cnLevelSelectionMode = "ExplicitLevels"
>> ; Levels = (/3.,4.,5.,6.5,8.,10.,12.,15.,18.,21.,25.,28.,34.,39.,45/)
>> ; res@cnLevels = Levels
>> res@cnFillMode = "RasterFill" ; activate raster mode
>> res@cnRasterSmoothingOn = True
>> res@lbLabelAutoStride = True
>> res@NoHeaderFooter = True
>> ;----------------------------------------------------------------------
>> ; Use WRF_contributed procedure to set map resources
>> ;----------------------------------------------------------------------
>> mpres = True
>> pltres = True
>> pltres@NoTitles = True
>> mpres@mpGeophysicalLineColor = 1
>> mpres@mpGridLineColor = 1
>> mpres@mpCenterLatF = a@CEN_LAT
>> mpres@mpCenterLonF = a@CEN_LON
>> ;----------------------------------------------------------------------
>> ; set True for native projection (faster)
>> ;----------------------------------------------------------------------
>>
>>
>> ; HOURLY LOOP OF PM 10 PLOTS
>>
>> do it = 0,23
>> itp1=it+1
>> print("I am working on hour "+itp1)
>>
>> ; INSERT EXAMPLE THICKNESS LENGTHS
>> thick = new((/27/),float) ; define a new array
>> thick = (/ 7.53, 26.39, 45.39, 72.2, 107.417, 120.2, 133.4, 159.0, 177.8,
>> 212.1, 213.2, 214.6, 263.764, 289.3, 326.5, 367.2, 402.1, 457.47, 809.45,
>> 916.6, 695.72, 750.3, 1049.4, 1282.5, 1582.8, 2874.39, 2875.0/)
>>
>> pm = a->PM10(it,:,:,:)
>> pm!0 = "lev"
>> pm!1 = "lat"
>> pm!2 = "lon"
>>
>> ;;;;;;;;;;;PROBLEM IS HERE;;;;;;;;;;;;;;;;
>> ; integrated = (thick) * (pm(:,0,0)) ;********
>> integrated = (thick) * (pm) ; *problem is here* ;*********
>> vertint = dim_sum(integrated(lat|:,lon|:,lev|:)) ;*********
>>
>> ; I understand here that the dimensions do not match, and I have solved
>> problems like this before. However, the latitude and longitude must be
>> included, as I am vertically integrating the PM10 with each grid-cell.
>>
>> ; However, I cannot create loops of "lat = 0,159" and "lon = 0,199". The
>> plots will take way too long to generate. I believe I need to multiply the
>> first dimension of "pm" by the thicknesses, but I am not sure how to do
>> so.
>>
>> ; Is there something simple that I am missing? I believe there is a much
>> simpler option. All I need to do is vertically integrate each layer, and I
>> already have the thicknesses to do so.
>>
>>
>> vertint@description="Vertically Integrated PM~B~10~N~ Concentration
>> "+itp1+"00 UTC "+monthname+" "+day+", "+year
>> vertint@units="~F33~m~F21~g ~N~m~S~-2~N~"
>>
>> ref_contour = wrf_contour(a,wks,vertint(5:154,5:194),res)
>> plot1 = wrf_map_overlays(a,wks,(/ref_contour/),pltres,mpres)
>>
>> end do
>> end
>>
>>
>> Thank you much once again.
>>
>> Mike
>>
>>
>>
>> _______________________________________________
>> ncl-talk mailing list
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>
>
> ------------------------------
>
> Message: 3
> Date: Thu, 27 Mar 2014 07:21:23 -0700 (PDT)
> From: Lisa Alexander <lisa_s_a@yahoo.ca>
> Subject: Re: Fatal Error Message
> To: Dennis Shea <shea@ucar.edu>
> Cc: "ncl-talk@ucar.edu" <ncl-talk@ucar.edu>
> Message-ID:
> <1395930083.76509.YahooMailBasic@web121906.mail.ne1.yahoo.com>
> Content-Type: text/plain; charset=iso-8859-1
>
> Hi Dennis,
> Thanks for the input. The array size is not any larger, it just loops through finding it many more times.
> I have 6 GB of memory on my computer. I am not sure what my variables require. But it loops through finding the values it requires many times before it has a problem. So, there is something building up over time, but it does not make sense to me because all variables are overwritten on the next loop.
>
> One thing that I am doing is appending the results for each loop to a .txt file. Could this be causing some problem?
> I attempted to run the program as you specified below, but it does not seem to identify the 'top' command.
>
> Thanks very much for your help,
> Lisa
> --------------------------------------------
> On Thu, 3/27/14, Dennis Shea <shea@ucar.edu> wrote:
>
> Subject: Re: Fatal Error Message
> To: "Lisa Alexander" <lisa_s_a@yahoo.ca>
> Cc: "ncl-talk@ucar.edu" <ncl-talk@ucar.edu>
> Received: Thursday, March 27, 2014, 9:20 AM
>
>
> [1] fatal:NclMalloc Failed:[errno=12]
>
> ???"I have been running a program on my
> computer, but this time
> ? ? the case time length is very long ..."
>
> ? ? I assume this means the array size(s) are much
> larger.
>
> ? ? You have run out of memory.
>
> ? ? If the primary variable is (say) 6GB and you
> have only 4GB,
> ? ? you will get a 'NclMalloc Failed' error. This
> is regardless
> ? ? if you have deleted all other variables.
>
> ? ? Do you know how much memory your computer has
> *available*?
> ? ? Do you know how much memory your main
> variable(s) need?
>
> [2] One crude approach
>
> ? ? %> ncl lisaAlexander.ncl >&! z.out
> &
> ? ? Then immediately,
> ? ? %> top
>
> ? ???This will show you the memory usage
> of your job.
>
>
> On 3/26/14, 4:12 PM, Lisa Alexander wrote:
> > Hello,
> > Please see the problem I have been experiencing below
> (noted in a previous post). I have tried to delete all
> variables once they are not being used and this does not
> work (does not improve the situation even slightly).
> > Would it be possible for someone to take a look at the
> program to see if there are any obvious sinks for computer
> memory?
> > Thanks for your help,
> > Lisa
> >
> > --- On Wed, 3/26/14, Lisa Alexander <lisa_s_a@yahoo.ca>
> wrote:
> >
> >> From: Lisa Alexander <lisa_s_a@yahoo.ca>
> >> Subject: Fatal Error Message
> >> To: ncl-talk@ucar.edu
> >> Received: Wednesday, March 26, 2014, 2:56 PM
> >> Hello,
> >> I have been running a program on my computer, but
> this time
> >> the case time length is very long. When I run it,
> it stops
> >> part way through and gives me this message:
> >>
> >> fatal:NclMalloc Failed:[errno=12]
> >> Segmentation fault (core dumped)
> >>
> >> I must say, whenever I run the program (this time
> and other
> >> times), the program runs very quickly to begin
> with, but
> >> slows down over time.
> >>
> >> Can somebody let me know what the error message
> means?
> >> Thanks,
> >> Lisa
> >> _______________________________________________
> >> ncl-talk mailing list
> >> List instructions, subscriber options,
> unsubscribe:
> >> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
> >>
> > _______________________________________________
> > ncl-talk mailing list
> > List instructions, subscriber options, unsubscribe:
> > http://mailman.ucar.edu/mailman/listinfo/ncl-talk
> >
>
>
>
>
> ------------------------------
>
> Message: 4
> Date: Thu, 27 Mar 2014 08:22:32 -0600
> From: "Ghassan J. Alaka, Jr." <gusalaka@atmos.colostate.edu>
> Subject: More than 2 Dimensions for "fft2df "
> To: ncl-talk@ucar.edu
> Message-ID:
> <CAMFLk_iUpLt0WuFs_RZ-+VsBcJNuTbssLBVnU2syzn3B_NAeNQ@mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Good morning,
>
> I'm running into an efficiency issue with my NCL script. I'm trying to
> filter time and longitude using "fft2df", but the variable that I want to
> filter contains 4 total dimensions (time, level, lat, lon), so I am
> currently looping over level and lat. then filtering each 2d time-lon slice
> using fft2df. This, of course, takes a long time when there are 37 levels
> and 86 latitudes (plus I need to do this for 20 different variables!).
>
> Is there any way to feed more than 2 dimensions to fft2df, with time and
> lon as the 2 rightmost dimensions? If not, is there another way I can
> filter using FFT and not have to loop over other dimensions? Perhaps there
> is a way to use "wgt_runave" to filter both time and longitude using a
> boxcar filter?
>
>
> Below is the code snippet where I loop (I forgot to mention that I also
> have to loop over several daily files to build the time series in the fist
> place):
> * do kk = 0,levsz-1*
> * do jj = 0,latsz-1*
> * do ff = 0,filesz-1*
> * f = addfile(filelist(ff),"r")*
> * Vfft((ntpd*ff)+0:(ntpd*ff)+(ntpd-1),:) \*
> * = (/f->$Vars(vv)$(:,kk,{lat(jj)},:)/)*
> * end do*
> * print(" Filtering selection "+kk+" "+jj)*
> * Vfft = kf_box_filter(Vfft, ntpd, tMin, tMax, kMin, kMax, inFlag)*
> * V(:,kk,jj,:) = (/Vfft(:,{sub_lon(0):sub_lon(1)})/)*
> * end do*
> * end do*
>
> The function "kf_box_filter.ncl" is adapted from kf_filter.ncl (I removed
> the options for filtering along specific dispersion curves for various
> equatorial waves). I've attached that here so you can see how I am calling
> fft2df... nothing complicated.
>
> Thanks for your help!
> Gus
>

-- 
Jennifer Beale
Ph. D. Candidate, Atmospheric Science
Department of Earth and Space Science, York University, Toronto
Petrie Building, 416-736-2100 Ext 30278
www.jennbeale.com
_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk