Re: [problems] satellite data -regridding ESMF

From: Dennis Shea <shea_at_nyahnyahspammersnyahnyah>
Date: Wed Jul 09 2014 - 10:51:33 MDT

Attached are two scripts:

[1] djs_v1.teste-regridding-tos.ncl

This uses ESMF regridding.

NOTE:This will fail with the following error (see: out_djs_v1)

 NetCDF Error: bilinear.4096x8192_to_224x256.nc : NetCDF: One or more
variable sizes violate format constraints

This is an issue with the ESMF netCDF and not, per se, with NCL.
A question has been sent to the ESMF project.

===
[2] Since you are going from one rectilinear grid (4096,8192) to another
(224,256) you can use linint2_Wrap
(see: djs_v2.linint2.ncl). This is simpler and faster. It will create a
netCDF file named 'teste.linint2.nc'

===
Good luck


On Tue, Jul 8, 2014 at 11:45 AM, VanĂșcia Schumacher <
vanucia-schumacher@hotmail.com> wrote:

> Hi Dennis,
>
> I did what you suggested, and returns the following error, the script
> fails to print the line: printVarSummary(var_regrid)
> print("var_regrid: min="+min(var_regrid)+" max="+max(var_regrid))
>
>
>
> Variable: var
> Type: float
> Total Size: 402653184 bytes
> 100663296 values
> Number of Dimensions: 3
> Dimensions and sizes: [ncl_join | 3] x [lat | 4096] x [lon | 8192]
> Coordinates:
> Number Of Attributes: 2
> units : Temp
> long_name : Temperature
> (0) var: min=-3 max=32.475
> (0) get_src_grid_info: source lat dims = (4096)
> (0) get_src_grid_info: source lon dims = (8192)
> (0) get_src_grid_info: source grid type is 'rectilinear'
> (0) curvilinear_to_SCRIP: calculating grid corners...
> fatal:NclMalloc Failed:[errno=12]
> fatal:New: could not create new array:[errno=12]
>
>
> Variable: src_lat
> Type: double
> Total Size: 32768 bytes
> 4096 values
> Number of Dimensions: 1
> Dimensions and sizes: [dim1 | 4096]
> Coordinates:
> Number Of Attributes: 14
> bsst.dsp_PixelType : 1
> bsst.dsp_PixelSize : 2
> bsst.dsp_Flag : 0
> bsst.dsp_nBits : 16
> bsst.dsp_LineSize : 0
> bsst.dsp_cal_name : Temperature
> bsst.units : Temp
> bsst.dsp_cal_eqnNumber : 2
> bsst.dsp_cal_CoeffsLength : 8
> bsst.dsp_cal_coeffs : ( 0.075, -3 )
> bsst.scale_factor : 0.075
> bsst.add_off : -3
> dim_0.name : lat
> dim_0.long_name : latitude
>
> Variable: src_lon
> Type: double
> Total Size: 65536 bytes
> 8192 values
> Number of Dimensions: 1
> Dimensions and sizes: [dim2 | 8192]
> Coordinates:
> Number Of Attributes: 14
> bsst.dsp_PixelType : 1
> bsst.dsp_PixelSize : 2
> bsst.dsp_Flag : 0
> bsst.dsp_nBits : 16
> bsst.dsp_LineSize : 0
> bsst.dsp_cal_name : Temperature
> bsst.units : Temp
> bsst.dsp_cal_eqnNumber : 2
> bsst.dsp_cal_CoeffsLength : 8
> bsst.dsp_cal_coeffs : ( 0.075, -3 )
> bsst.scale_factor : 0.075
> bsst.add_off : -3
> dim_1.name : lon
> dim_1.long_name : longitude
>
> begin
> fils = systemfunc("ls *.nc")
> f = addfiles(fils, "r")
> ListSetType(f,"join")
>
> var_short=f[:]->bsst ; f[0]->bsst
>
> var = var_short*var_short@scale_factor + var_short@add_off
> copy_VarCoords(var_short, var) ; copy coordinate information
>
> var@long_name = var_short@dsp_cal_name
> var@units = var_short@units
>
> printVarSummary(var)
> print("var: min="+min(var)+" max="+max(var))
>
> src_lat = f[0]->lat
> src_lon = f[0]->lon
> printVarSummary(src_lat)
> printVarSummary(src_lon)
> newfile = "teste.nc"
> system("rm -f "+newfile)
> ncdf = addfile(newfile ,"c")
> setvalues NhlGetWorkspaceObjectId
> "wsMaximumSize" : 300000000
> end setvalues
> Opt = True
> Opt@SrcGridLat = src_lat ; source grid
> Opt@SrcGridLon = src_lon
> newlat = fspan( -89.7, 89.7,224)
> newlon = fspan(0.0,359.7,256)
> tGridType = "rectilinear"
> Opt@DstGridLat = newlat
> Opt@DstGridLon = newlon
> Opt@DstFileName = "Rectilinear.nc" ; destination files
> Opt@InterpMethod = "bilinear"
> Opt@ForceOverwrite = True
> Opt@PrintTimings = True
> Opt@Debug = True
>
> var_regrid = ESMF_regrid(var,Opt)
> printVarSummary(var_regrid)
> print("var_regrid: min="+min(var_regrid)+" max="+max(var_regrid))
>
> var_regrid!0 ="lat"
> var_regrid!1 = "lon"
> var_regrid&lat = newlat
> var_regrid&lat@units="degrees_north"
> var_regrid&lon = newlon
> var_regrid&lon@units="degrees_east"
> ncdf->bsst = var_regrid
> end
>
> ---
> VanĂșcia Schumacher
> Mestranda em Meteorologia - UFV
> Meteorologista -UFPel
> Departamento de Meteorologia AgrĂ­cola - DEA
> Cel: (31) 9978 2522
> DEA: (31) 3899 1890
>
>
> ------------------------------
> Date: Mon, 7 Jul 2014 21:54:08 -0600
> Subject: Re: [problems] satellite data -regridding ESMF
> From: shea@ucar.edu
> To: vanucia-schumacher@hotmail.com
> CC: ncl-talk@ucar.edu
>
> The 1st rule of data processing is to look at your data.
>
> There may be several issues.
>
> [1] Given the dimensions [lat | 4096] x [lon | 8192],
> I speculate the same grid is used for each file.
>
> Change:
>
> src_lat = f[:]->lat
> src_lon = f[:]->lon
> to
> src_lat = f[0]->lat ; get lat/lon from 1st file
> src_lon = f[0]->lon
>
> take a look at the variables. Does it look reasonable to you???
>
> printVarSummary(src_lat)
> printVarSummary(src_lon)
>
> [2] If what I said in [1] is true then you need only one input file
> to generate the weight file. The others contain redundant lat/lon
> information.
>
>
> [3] The variable is type short and it *must* be unpacked prior to use.
> Unfortunately, one of the attributes is non-standard and is not
> recognized
> by NCL' 'short2flt' function.
>
> scale_factor : 0.075 <=== standard attribute
> add_off : -3 <=== non-standard (add_offset =
would
> be standard)
>
> This means the user must explicitly perform the unpacking and handl=
e
> the
> meta data
>
>
> var_short=f[:]->bsst ; f[0]->bsst
>
> var = var_short*var_short@scale_factor + var_short@add_off
> copy_VarCoords(var_short, var) ; copy coordinate information
>
> var@long_name = var_short@dsp_cal_name
> var@units = var_short@units
>
> Look at the variable
>
> printVarSummary(var)
> print("var: min="+min(var)+" max="+max(var))
>
> [4]
> var_regrid = ESMF_regrid(var,Opt)
> printVarSummary( var_regrid )
> print("var_regrid: min="+min(var_regrid)+"
> max="+max(var_regrid))
>
> Likely ... there are other changes needed but this gets you started.
>
>
>
> On Mon, Jul 7, 2014 at 3:49 PM, VanĂșcia Schumacher <
> vanucia-schumacher@hotmail.com> wrote:
>
> Hi users,
>
> I'm using satellite data, but when I try to run the script to change the
> grid (esmf), returns the errors:
>
> (0) get_src_grid_info: source lat dims = (3,4096)
> (0) get_src_grid_info: source lon dims = (3,8192)
> (0) get_src_grid_info: SrcGridType and/or SrcGridLat/SrcGridLon were not
> set.
> (0) Cannot determine the source grid type.
>
> Variable: var
> Type: short
> Total Size: 201326592 bytes
> 100663296 values
> Number of Dimensions: 3
> Dimensions and sizes: [ncl_join | 3] x [lat | 4096] x [lon | 8192]
> Coordinates:
> Number Of Attributes: 28
> bsst.dsp_PixelType : 1
> bsst.dsp_PixelSize : 2
> bsst.dsp_Flag : 0
> bsst.dsp_nBits : 16
> bsst.dsp_LineSize : 0
> bsst.dsp_cal_name : Temperature
> bsst.units : Temp
> bsst.dsp_cal_eqnNumber : 2
> bsst.dsp_cal_CoeffsLength : 8
> bsst.dsp_cal_coeffs : ( 0.075, -3 )
> bsst.scale_factor : 0.075
> bsst.add_off : -3
> dsp_PixelType : 1
> dsp_PixelSize : 2
> dsp_Flag : 0
> dsp_nBits : 16
> dsp_LineSize : 0
> dsp_cal_name : Temperature
> units : Temp
> dsp_cal_eqnNumber : 2
> dsp_cal_CoeffsLength : 8
> dsp_cal_coeffs : ( 0.075, -3 )
> scale_factor : 0.075
> add_off : -3
> dim_0.name : lat
> dim_0.long_name : latitude
> dim_1.name : lon
> dim_1.long_name : longitude
>
>
> begin
> fils = systemfunc("ls *.nc")
> f = addfiles(fils, "r")
> ListSetType(f,"join")
> var=f[:]->bsst
> printVarSummary(var)
> src_lat = f[:]->lat
> src_lon = f[:]->lon
> newfile = "teste.nc"
> system("rm -f "+newfile)
> ncdf = addfile(newfile ,"c") ; open output netCDF file
> setvalues NhlGetWorkspaceObjectId
> "wsMaximumSize" : 300000000
> end setvalues
> Opt = True
> Opt@SrcGridLat = src_lat ; source grid
> Opt@SrcGridLon = src_lon
> ;Opt@SrcMask2D = maskfile->MASKSURF;
> newlat = fspan( -89.7, 89.7,224)
> newlon = fspan(0.0,359.7,256)
> tGridType = "rectilinear"
> Opt@DstGridLat = newlat
> Opt@DstGridLon = newlon
> Opt@DstFileName = "Rectilinear.nc" ; destination files
> Opt@InterpMethod = "bilinear"
> Opt@ForceOverwrite = True
> Opt@PrintTimings = True
> Opt@Debug = True
> var_regrid = ESMF_regrid(var,Opt) ; Do the regridding
>
> var_regrid!0 ="time"
> var_regrid!1 ="lat"
> var_regrid!2 = "lon"
> var_regrid&time = var&time
> var_regrid&lat = newlat
> var_regrid&lat@units="degrees_north"
> var_regrid&lon = newlon
> var_regrid&lon@units="degrees_east"
>
> ncdf->time_bnds = f[:]->time_bnds
> ncdf->bsst = var_regrid
> end
>
>
> ---
> VanĂșcia Schumacher
> Mestranda em Meteorologia - UFV
> Meteorologista -UFPel
> Departamento de Meteorologia AgrĂ­cola - DEA
> Cel: (31) 9978 2522
> DEA: (31) 3899 1890
>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
>

Received on Wed Jul 09 04:51:42 2014

This archive was generated by hypermail 2.1.8 : Wed Jul 23 2014 - 15:33:46 MDT