Re: Segmentation error when regridding

Hi all,

After some testing it was determined that Will was brushing up against
NCL's single data array size limit, which is somewhere near 4GB. Will is
working on reducing the size of his array by only reading in a few years
at a time instead of all 26 years at once.

So you all know, raising the 4GB array size limit is on the NCL
developers priority list.
Adam

Will Hobbs wrote:
> Hi
>
> I am running the following script on tempest, which regrids a GODAS data
> file, first in the vertical and then horizontal. I keep getting
> Segmentation errors, which we thought initially were due to limited
> stacksize, and so I've added loops into the regridding to reduce the
> array size being processed at any onoe time - still getting Segmentation
> errors. I know from the print statements that it fails during the first
> regrid (i.e. interpolating depth).
>
> Any ideas where Ican go from here? Is there anything wrong with the script?
>
> (The script and data files are all on tempest at /ptmp/whobbs/)
>
>
> begin
>
> ;data file locations
> Diri0 = ""
>
> Fil1 = "godas_u_1980_2005.nc"
> Fil2 = "godas_dzdt_1980_2005.nc"
>
>
> ;*******************************************
> ;Read in data
> ;*****************************************
>
> print("reading w data")
>
>
> ;read in grid data
>
> f1 = addfile(Diri0+Fil1, "r")
> depth = f1->depth
>
> f2 = addfile(Diri0+Fil2, "r")
> tlon = f2->tlon
> tlat = f2->tlat({-90.:0.})
> DEPTH = f2->DEPTH
> time = f2->time
>
> x = f2->DZDT(:,:,{-90.:0.},:)
>
> scale = x_at_scale_factor ;convert to float (error in short2flt)
> offs = x_at_add_offset
>
>
> print(scale)
> print(offs)
>
> w = (x * scale) + offs
>
>
> ;copy required attributes
>
> wAtt = 0.
> wAtt@_FillValue = short2flt(x@_FillValue)
> wAtt_at_units = x_at_units
> wAtt_at_long_name = x_at_long_name
> wAtt_at_vMin_user_specified = x_at_vMin_user_specified
> wAtt_at_vMax_user_specified = x_at_vMax_user_specified
> wAtt_at_vRange = x_at_vRange
>
> delete(x)
>
>
> ;regrid vertical
>
> print("regridding w (z)")
>
>
> w!0 = "time" ;dim names for reordering
> w!1 = "DEPTH"
> w!2 = "tlat"
> w!3 = "tlon"
>
>
> ntim = dimsizes(time) ;dim sizes for new array
> ndepth = dimsizes(depth)
> ntlat = dimsizes(tlat)
> ntlon = dimsizes(tlon)
>
>
> w_1 = new((/ ntim, ntlat, ntlon, ndepth/), typeof(w))
>
>
>
> do tt = 0, ntim-1
>
> w_1(tt,:,:,:) = (/ linint1(DEPTH, w(time|tt, tlat|:, tlon|:,
> DEPTH|:), Fa
> lse, depth, 0) /)
>
> end do
>
> delete(w)
> delete(DEPTH)
>
>
>
> ;regrid horizontal
>
> print("regridding w (x & y)")
>
> w_1!0 = "time" ;dim names for reordering
> w_1!1 = "depth"
> w_1!2 = "tlat"
> w_1!3 = "tlon"
>
>
> ulon = f1->ulon ;new lat/lon coord arrays
> ulat = f1->ulat({-90.:0.})
>
> nulat = dimsizes(ulat) ;new array dimsizes
> nulon = dimsizes(ulon)
>
> w_2 = new((/ntim, ndepth, nulat, nulon/), typeof(w_1))
>
>
> do zz = 0, ndepth-1
>
> w_2(:,zz,:,:) = (/ linint2(tlon(TXMin:TXMax),
> tlat(TYMin:TYMax), w_1(t
> ime|:, depth|zz, tlat|:, tlon|:), True, ulon(UXMin:UXMax),
> ulat(UYMin:UYMax), 0)
> /)
>
> end do
>
>
> delete(w_1)
> delete(tlat)
> delete(tlon)
>
>
> printVarSummary(w_2)
>
>
>
>
> ;********************************************
> ;Write w data to file
> ;******************************************
>
> print("creating w file")
>
> Fil4 = "godas_dzdt_1980_2005_regrid.nc"
>
>
> ;w file
>
> system("rm "+diri0+FIL4)
>
> fw = addfile(diri0+FIL4, "c")
>
> ;file attributes
>
> fAtt = True
> fAtt_at_title = "GODAS DZDT"
> fAtt_at_source_file = FIL2
> fAtt_at_conventions = "none"
> fAtt_at_creation_date = systemfunc ("date")
>
> fileattdef( fw, fAtt)
>
> ;coordinates
>
> wdimz = dimsizes(w_2)
>
> dimNames = (/"time","depth", "ulat", "ulon"/)
> dimLength = (/-1, wdimz(1), wdimz(2), wdimz(3)/)
> dimUnlim = (/True, False, False, False/)
>
> filedimdef(fw, dimNames, dimLength, dimUnlim)
>
>
> ;file variables
>
> filevardef(fw, "time", typeof(time), "time")
> filevardef(fw, "depth", typeof(depth), "depth")
> filevardef(fw, "ulat", typeof(ulat), "ulat")
> filevardef(fw, "ulon", typeof(ulon), "ulon")
> filevardef(fw, "DZDT", typeof(w_2), (/"time","depth","ulat","ulon"/))
>
>
> ;variable attributes
>
> filevarattdef(fw, "time", time)
> filevarattdef(fw, "depth", depth)
> filevarattdef(fw, "ulat", ulat)
> filevarattdef(fw, "ulon", ulon)
> filevarattdef(fw, "DZDT", wAtt)
>
>
>
> ;copy data to file
>
> fw->time = (/time/)
> fw->depth = (/depth/)
> fw->ulat = (/ulat/)
> fw->ulon = (/ulon/)
> fw->DZDT = (/w_2/)
>
> delete(w_2)
>
> end
>
> ============================
> Room 3221C
> UCLA Department of Geography
> 1255 Bunche Hall
> Los Angeles CA 90095-1524
>
>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk_at_ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk

-- 
--------------------------------------------------------------
Adam Phillips			             asphilli_at_ucar.edu
National Center for Atmospheric Research   tel: (303) 497-1726
ESSL/CGD/CAS                               fax: (303) 497-1333
P.O. Box 3000				
Boulder, CO 80307-3000	  http://www.cgd.ucar.edu/cas/asphilli
_______________________________________________
ncl-talk mailing list
ncl-talk_at_ucar.edu
http://mailman.ucar.edu/mailman/listinfo/ncl-talk