Re: [problems] satellite data -regridding ESMF

The following
---
Fatal:NclMalloc Failed:[errno=12] <=============
======
fatal:New: could not create new array:[errno=12]
---
is telling you that you have exceed available memory.

I think it would be best to make one of your data files available via one
of the following:

(a) place the file some place from which we can download it

(b) tell us a URL where the dataset resides

(c) ftp the file to NCAR and, after successful transfer, send us the name
of the file.

ftp ftp.cgd.ucar.edu
anonymous
<use your email address for the password>
cd incoming
put ...
quit

 Note that for security reasons, you cannot list the contents of this
directory, and neither can we. We need to know the *exact* names of the
file in order to retrieve it.




On Tue, Jul 8, 2014 at 11:45 AM, Vanúcia Schumacher <
vanucia-schumacher@hotmail.com> wrote:

> Hi Dennis,
>
> I did what you suggested, and returns the following error, the script
> fails to print the line: printVarSummary(var_regrid)
> print("var_regrid: min="+min(var_regrid)+" max="+max(var_regrid))
>
>
>
> Variable: var
> Type: float
> Total Size: 402653184 bytes
> 100663296 values
> Number of Dimensions: 3
> Dimensions and sizes: [ncl_join | 3] x [lat | 4096] x [lon | 8192]
> Coordinates:
> Number Of Attributes: 2
> units : Temp
> long_name : Temperature
> (0) var: min=-3 max=32.475
> (0) get_src_grid_info: source lat dims = (4096)
> (0) get_src_grid_info: source lon dims = (8192)
> (0) get_src_grid_info: source grid type is 'rectilinear'
> (0) curvilinear_to_SCRIP: calculating grid corners...
> fatal:NclMalloc Failed:[errno=12]
> fatal:New: could not create new array:[errno=12]
>
>
> Variable: src_lat
> Type: double
> Total Size: 32768 bytes
> 4096 values
> Number of Dimensions: 1
> Dimensions and sizes: [dim1 | 4096]
> Coordinates:
> Number Of Attributes: 14
> bsst.dsp_PixelType : 1
> bsst.dsp_PixelSize : 2
> bsst.dsp_Flag : 0
> bsst.dsp_nBits : 16
> bsst.dsp_LineSize : 0
> bsst.dsp_cal_name : Temperature
> bsst.units : Temp
> bsst.dsp_cal_eqnNumber : 2
> bsst.dsp_cal_CoeffsLength : 8
> bsst.dsp_cal_coeffs : ( 0.075, -3 )
> bsst.scale_factor : 0.075
> bsst.add_off : -3
> dim_0.name : lat
> dim_0.long_name : latitude
>
> Variable: src_lon
> Type: double
> Total Size: 65536 bytes
> 8192 values
> Number of Dimensions: 1
> Dimensions and sizes: [dim2 | 8192]
> Coordinates:
> Number Of Attributes: 14
> bsst.dsp_PixelType : 1
> bsst.dsp_PixelSize : 2
> bsst.dsp_Flag : 0
> bsst.dsp_nBits : 16
> bsst.dsp_LineSize : 0
> bsst.dsp_cal_name : Temperature
> bsst.units : Temp
> bsst.dsp_cal_eqnNumber : 2
> bsst.dsp_cal_CoeffsLength : 8
> bsst.dsp_cal_coeffs : ( 0.075, -3 )
> bsst.scale_factor : 0.075
> bsst.add_off : -3
> dim_1.name : lon
> dim_1.long_name : longitude
>
> begin
> fils = systemfunc("ls *.nc")
> f = addfiles(fils, "r")
> ListSetType(f,"join")
>
> var_short=f[:]->bsst ; f[0]->bsst
>
> var = var_short*var_short@scale_factor + var_short@add_off
> copy_VarCoords(var_short, var) ; copy coordinate information
>
> var@long_name = var_short@dsp_cal_name
> var@units = var_short@units
>
> printVarSummary(var)
> print("var: min="+min(var)+" max="+max(var))
>
> src_lat = f[0]->lat
> src_lon = f[0]->lon
> printVarSummary(src_lat)
> printVarSummary(src_lon)
> newfile = "teste.nc"
> system("rm -f "+newfile)
> ncdf = addfile(newfile ,"c")
> setvalues NhlGetWorkspaceObjectId
> "wsMaximumSize" : 300000000
> end setvalues
> Opt = True
> Opt@SrcGridLat = src_lat ; source grid
> Opt@SrcGridLon = src_lon
> newlat = fspan( -89.7, 89.7,224)
> newlon = fspan(0.0,359.7,256)
> tGridType = "rectilinear"
> Opt@DstGridLat = newlat
> Opt@DstGridLon = newlon
> Opt@DstFileName = "Rectilinear.nc" ; destination files
> Opt@InterpMethod = "bilinear"
> Opt@ForceOverwrite = True
> Opt@PrintTimings = True
> Opt@Debug = True
>
> var_regrid = ESMF_regrid(var,Opt)
> printVarSummary(var_regrid)
> print("var_regrid: min="+min(var_regrid)+" max="+max(var_regrid))
>
> var_regrid!0 ="lat"
> var_regrid!1 = "lon"
> var_regrid&lat = newlat
> var_regrid&lat@units="degrees_north"
> var_regrid&lon = newlon
> var_regrid&lon@units="degrees_east"
> ncdf->bsst = var_regrid
> end
>
> ---
> Vanúcia Schumacher
> Mestranda em Meteorologia - UFV
> Meteorologista -UFPel
> Departamento de Meteorologia Agrícola - DEA
> Cel: (31) 9978 2522
> DEA: (31) 3899 1890
>
>
> ------------------------------
> Date: Mon, 7 Jul 2014 21:54:08 -0600
> Subject: Re: [problems] satellite data -regridding ESMF
> From: shea@ucar.edu
> To: vanucia-schumacher@hotmail.com
> CC: ncl-talk@ucar.edu
>
> The 1st rule of data processing is to look at your data.
>
> There may be several issues.
>
> [1] Given the dimensions [lat | 4096] x [lon | 8192],
> I speculate the same grid is used for each file.
>
> Change:
>
> src_lat = f[:]->lat
> src_lon = f[:]->lon
> to
> src_lat = f[0]->lat ; get lat/lon from 1st file
> src_lon = f[0]->lon
>
> take a look at the variables. Does it look reasonable to you???
>
> printVarSummary(src_lat)
> printVarSummary(src_lon)
>
> [2] If what I said in [1] is true then you need only one input file
> to generate the weight file. The others contain redundant lat/lon
> information.
>
>
> [3] The variable is type short and it *must* be unpacked prior to use.
> Unfortunately, one of the attributes is non-standard and is not
> recognized
> by NCL' 'short2flt' function.
>
> scale_factor : 0.075 <=== standard attribute
> add_off : -3 <=== non-standard (add_offset =
would
> be standard)
>
> This means the user must explicitly perform the unpacking and handl=
e
> the
> meta data
>
>
> var_short=f[:]->bsst ; f[0]->bsst
>
> var = var_short*var_short@scale_factor + var_short@add_off
> copy_VarCoords(var_short, var) ; copy coordinate information
>
> var@long_name = var_short@dsp_cal_name
> var@units = var_short@units
>
> Look at the variable
>
> printVarSummary(var)
> print("var: min="+min(var)+" max="+max(var))
>
> [4]
> var_regrid = ESMF_regrid(var,Opt)
> printVarSummary( var_regrid )
> print("var_regrid: min="+min(var_regrid)+"
> max="+max(var_regrid))
>
> Likely ... there are other changes needed but this gets you started.
>
>
>
> On Mon, Jul 7, 2014 at 3:49 PM, Vanúcia Schumacher <
> vanucia-schumacher@hotmail.com> wrote:
>
> Hi users,
>
> I'm using satellite data, but when I try to run the script to change the
> grid (esmf), returns the errors:
>
> (0) get_src_grid_info: source lat dims = (3,4096)
> (0) get_src_grid_info: source lon dims = (3,8192)
> (0) get_src_grid_info: SrcGridType and/or SrcGridLat/SrcGridLon were not
> set.
> (0) Cannot determine the source grid type.
>
> Variable: var
> Type: short
> Total Size: 201326592 bytes
> 100663296 values
> Number of Dimensions: 3
> Dimensions and sizes: [ncl_join | 3] x [lat | 4096] x [lon | 8192]
> Coordinates:
> Number Of Attributes: 28
> bsst.dsp_PixelType : 1
> bsst.dsp_PixelSize : 2
> bsst.dsp_Flag : 0
> bsst.dsp_nBits : 16
> bsst.dsp_LineSize : 0
> bsst.dsp_cal_name : Temperature
> bsst.units : Temp
> bsst.dsp_cal_eqnNumber : 2
> bsst.dsp_cal_CoeffsLength : 8
> bsst.dsp_cal_coeffs : ( 0.075, -3 )
> bsst.scale_factor : 0.075
> bsst.add_off : -3
> dsp_PixelType : 1
> dsp_PixelSize : 2
> dsp_Flag : 0
> dsp_nBits : 16
> dsp_LineSize : 0
> dsp_cal_name : Temperature
> units : Temp
> dsp_cal_eqnNumber : 2
> dsp_cal_CoeffsLength : 8
> dsp_cal_coeffs : ( 0.075, -3 )
> scale_factor : 0.075
> add_off : -3
> dim_0.name : lat
> dim_0.long_name : latitude
> dim_1.name : lon
> dim_1.long_name : longitude
>
>
> begin
> fils = systemfunc("ls *.nc")
> f = addfiles(fils, "r")
> ListSetType(f,"join")
> var=f[:]->bsst
> printVarSummary(var)
> src_lat = f[:]->lat
> src_lon = f[:]->lon
> newfile = "teste.nc"
> system("rm -f "+newfile)
> ncdf = addfile(newfile ,"c") ; open output netCDF file
> setvalues NhlGetWorkspaceObjectId
> "wsMaximumSize" : 300000000
> end setvalues
> Opt = True
> Opt@SrcGridLat = src_lat ; source grid
> Opt@SrcGridLon = src_lon
> ;Opt@SrcMask2D = maskfile->MASKSURF;
> newlat = fspan( -89.7, 89.7,224)
> newlon = fspan(0.0,359.7,256)
> tGridType = "rectilinear"
> Opt@DstGridLat = newlat
> Opt@DstGridLon = newlon
> Opt@DstFileName = "Rectilinear.nc" ; destination files
> Opt@InterpMethod = "bilinear"
> Opt@ForceOverwrite = True
> Opt@PrintTimings = True
> Opt@Debug = True
> var_regrid = ESMF_regrid(var,Opt) ; Do the regridding
>
> var_regrid!0 ="time"
> var_regrid!1 ="lat"
> var_regrid!2 = "lon"
> var_regrid&time = var&time
> var_regrid&lat = newlat
> var_regrid&lat@units="degrees_north"
> var_regrid&lon = newlon
> var_regrid&lon@units="degrees_east"
>
> ncdf->time_bnds = f[:]->time_bnds
> ncdf->bsst = var_regrid
> end
>
>
> ---
> Vanúcia Schumacher
> Mestranda em Meteorologia - UFV
> Meteorologista -UFPel
> Departamento de Meteorologia Agrícola - DEA
> Cel: (31) 9978 2522
> DEA: (31) 3899 1890
>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
>

Received on Tue Jul 08 06:21:07 2014