Hi Robert.
Thanks for the explanation, that makes complete sense. It is now
clear that the NCL user must always create an SrcMask2D input mask for
ESMF_regrid, whenever the input contains any missing values. This was
already in in the function documentation, and I missed it.
I would like to suggest that NCL examples ESMF_regrid_1, daymet_2, and
daymet_3 be updated to demonstrate the correct usage of SrcMask2D in
the presence of missing input values. I think this will take care of
the bathtub ring effects that I mentioned earlier.
Also please consider adding a warning to the ESMF_regrid function,
when the input contains the _FillValue attribute, and SrcMask2D is
missing. Thanks for your consideration.
--Dave
On Wed, Apr 17, 2013 at 11:23 AM, Robert Oehmke <robert.oehmke@noaa.gov> wrote:
> Hi Dave,
>
> The ESMF regrid weight calculation code in NCL doesn't have any access to the user data, so it can't detect or act on missing values (the application of the interpolation weights is done by NCL). The ESMF code only has access to the grid files which the NCL code constructs for it to calculate the interpolation weight matrix. The user can set masks in the grids which are provided to ESMF to enable it to ignore locations with bad values, and thus not have them be part of the weight matrix. Because NCL is generating the grids for ESMF in this case, the masking would have to be set via NCL. Mary H. would be the best person to describe how to do this (I believe that she mentioned SrcMask2D earlier in the thread). I think that the best way to proceed is to try setting the masking and see if that helps with the problem.
>
> Let me know if you have any questions about the above.
>
> Thanks!
>
> - Bob
>
> On Apr 15, 2013, at 4:13 PM, Dave Allured - NOAA Affiliate wrote:
>
>> Hi Robert.
>>
>> Thanks for looking into this. At the NCL application level, it does
>> not appear that the first example uses any explicit masking. Judging
>> from the plot, it looks like the SST input data is previously masked,
>> with valid numbers over water, and missing values over land. Here is
>> the NCL code, and the before and after plots for the first example:
>>
>> http://www.ncl.ucar.edu/Applications/Scripts/ESMF_all_1.ncl
>> http://www.ncl.ucar.edu/Applications/Images/ESMF_all_1_lg.png
>>
>> The presence of large patches of missing values is common to all of
>> the other examples and cases in this current discussion thread.
>>
>> --Dave
>>
>> On Mon, Apr 15, 2013 at 10:26 AM, Robert Oehmke <robert.oehmke@noaa.gov> wrote:
>>> Hi,
>>>
>>> This is Bob Oehmke of the ESMF core team. I want to make sure that I understand what's going on to produce these images, so that I can help figure out what's causing the problem. Are you using masking to remove the missing source values from consideration?
>>>
>>> Thanks!
>>>
>>> - Bob
>>>
>>>
>>>
>>> On Apr 12, 2013, at 4:58 PM, Dave Allured - NOAA Affiliate wrote:
>>>
>>>> Ping's example is fairly dramatic, but I think I have seen this
>>>> bathtub ring effect in other cases:
>>>>
>>>> http://www.ncl.ucar.edu/Applications/Images/ESMF_all_1_lg.png
>>>> http://www.ncl.ucar.edu/Applications/Images/daymet_2_lg.png
>>>> http://www.ncl.ucar.edu/Applications/Images/daymet_3_lg.png
>>>>
>>>> All cases use ESMF regridding, and all cases display an unexpected
>>>> bias toward zero along the interface between data and missing values.
>>>> Something NOT in common is the conserve method; ESMF_all_1 uses ESMF
>>>> bilinear.
>>>>
>>>> I do not have a solution, but please look for the possibility that
>>>> input missing values are being interpreted as zeros in the core regrid
>>>> calculation. You would get the same problem if missing values are
>>>> accidentally being counted in the divisor, when computing the local
>>>> area average for each output grid point. HTH.
>>>>
>>>> --Dave
>>>>
>>>> On Fri, Apr 12, 2013 at 4:12 PM, Mary Haley <haley@ucar.edu> wrote:
>>>>> Hi Ping,
>>>>>
>>>>> If you have the three *.nc files that are created by ESMF_regrid, then the ESMF folks might be able to look at them to see if there's an issue with the conserve method.
>>>>> They would also need to see exactly how ESMF_RegridWeightGen is being called, which you can see if you set:
>>>>>
>>>>> Opt@Debug = True
>>>>>
>>>>> Or, if you can provide me with your two input data files, then I can narrow down the script so that it only does one timestamp, and I will send the info off to the ESMF folks.
>>>>>
>>>>> http://www.ncl.ucar.edu/report_bug.shtml#HowToFTP
>>>>>
>>>>> --Mary
>>>>>
>>>>>
>>>>> On Apr 12, 2013, at 3:19 PM, Ping Yang wrote:
>>>>>
>>>>>> Dear NCL,
>>>>>>
>>>>>> I was using the following script to regridding the joined Daymet grid (1 KM) to 3 acrmin, in which the projection was translated from LLC to lonlat and the resolution from 1 KM to 3 arcmin. However, after I tried to comparing different interpolation method based on the 3 arcmin data (I used a conserve method for the ESMF regridding), I found the this 3 arcmin data has problem in the edge/boundary.
>>>>>>
>>>>>> I am including a graphs here to show the problem:
>>>>>> <image.png>
>>>>>> The edge(coast and land) place especially in Maine (also massachusetts) looks weird after the regridding, a mixture of no data value and real values, and when I do a statistical of year sum, I got very small values(below 10 mm) for one year.
>>>>>>
>>>>>> The script I was using is:
>>>>>>
>>>>>> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"
>>>>>> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"
>>>>>> load "$NCARG_ROOT/lib/ncarg/nclscripts/wrf/WRFUserARW.ncl"
>>>>>> load "$NCARG_ROOT/lib/ncarg/nclscripts/esmf/ESMF_regridding.ncl"
>>>>>> vname ="pr"
>>>>>> srcFileName = vname + ".NE." + year +".nc"
>>>>>> ;srcFileName = "NE_all_1980_correct_prcp.nc"
>>>>>> srcPathName = srcDirName+srcFileName
>>>>>> sfile = addfile(srcPathName,"r")
>>>>>>
>>>>>> ;---Set the destination file name
>>>>>> dstDirName = "/data/ecr/yangping/DAYMET/Grid/Intermediate/"
>>>>>> system("/bin/mkdir -p "+ dstDirName)
>>>>>> dstFileName = str_upper(vname)+"."+year+"_1KM.esmf.nc"
>>>>>> dstPathName = dstDirName+dstFileName
>>>>>>
>>>>>> setfileoption("nc","Format","LargeFile")
>>>>>>
>>>>>> ;out=addfile("/data/ecr/yangping/DAYMET/Grid/3MIN/" + vname + "_3MIN_lonlat_" + year+".nc","c")
>>>>>> outDirName = "/data/ecr/yangping/DAYMET/Grid/3MIN/"
>>>>>> outFileName = vname + ".3MIN.lonlat." + year+".nc"
>>>>>> outPathName = outDirName + outFileName
>>>>>> system("/bin/rm -f "+outPathName) ; delete any pre-existing file
>>>>>> out=addfile(outPathName,"c")
>>>>>>
>>>>>> ;--- ESMF regrid method: "conserve", "bilinear", "patch"
>>>>>> method = "conserve"
>>>>>> ; method = "patch"
>>>>>>
>>>>>> ;===================================================
>>>>>>
>>>>>> ;---Options
>>>>>> Opt = True
>>>>>> Opt@SrcTitle = "Daymet Joined Grid "+year+" to rectlinear" ; optional
>>>>>> Opt@WgtFileName = dstDirName + "Daymet_to_Rect.Wgt.nc"
>>>>>>
>>>>>> Opt@ForceOverwrite = True
>>>>>> ;;Opt@PrintTimings = True
>>>>>>
>>>>>> ;---Get the source file Daymet lat/lon grid
>>>>>> lat2d = sfile->lat
>>>>>> lon2d = sfile->lon
>>>>>>
>>>>>> dim2d = dimsizes(lat2d)
>>>>>> nlat = dim2d(0)
>>>>>> nlon = dim2d(1)
>>>>>>
>>>>>> print(nlat)
>>>>>> print(nlon)
>>>>>>
>>>>>> ;---Get the Daymet source variable
>>>>>> var = sfile->$vname$ ; (time, y, x)=>(365,232,221)
>>>>>>
>>>>>> ;Opt@SrcGridType = "curvilinear"
>>>>>> Opt@SrcGridLat = lat2d
>>>>>> Opt@SrcGridLon = lon2d
>>>>>>
>>>>>> ;---Create the destination lat/lon grid
>>>>>> dll=0.05 ;3MIN
>>>>>> latrange=abs(max(lat2d)-min(lat2d))
>>>>>> lonrange=abs(max(lon2d)-min(lon2d))
>>>>>> kpts_lat= toint(latrange/dll)+1
>>>>>> kpts_lon= toint(lonrange/dll)+1
>>>>>> lat = fspan( min(lat2d),max(lat2d), kpts_lat )
>>>>>> lon = fspan( min(lon2d),max(lon2d), kpts_lon )
>>>>>>
>>>>>> newlon = fspan(-84.025,-64.875,384)
>>>>>> newlat = fspan(34.875,50.125,306)
>>>>>>
>>>>>> Opt@DstGridType = "rectilinear"
>>>>>> ;Opt@DstGridLat = lat
>>>>>> Opt@DstGridLat = newlat
>>>>>> ;Opt@DstGridLon = lon
>>>>>> Opt@DstGridLon = newlon
>>>>>>
>>>>>> Opt@CopyVarCoords= True ; default for 6.1.0
>>>>>>
>>>>>> Opt@InterpMethod = method
>>>>>> Opt@SrcRegional = True
>>>>>> Opt@DstRegional = True
>>>>>>
>>>>>> var_regrid = ESMF_regrid(var,Opt) ; Do the regridding for 'var'
>>>>>> printVarSummary(var_regrid)
>>>>>> var_regrid@grid_mapping = "lonlat" ;
>>>>>> out->$vname$=var_regrid ;output the regridding result
>>>>>>
>>>>>> Can some here where is the problem and how to fix it?
>>>>>>
>>>>>> Look forward to hearing from you.
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Ping
_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Thu Apr 18 21:07:47 2013
This archive was generated by hypermail 2.1.8 : Fri Apr 26 2013 - 17:10:26 MDT
