Re: Joining Daymet Tiles with NCL

From: Ping Yang <pyang_at_nyahnyahspammersnyahnyah>
Date: Sun Oct 28 2012 - 19:54:14 MDT

Hi NCL,

I tested the new script several times and I found it still didn't fix the
problem. I then exclude the tile 11937 and 11938 and it works well both
with the previous script and the updated one.

A problem came to the plotting, which I subset a day (e.g. April 8 2010)
and the plotting looks like this(no color filled in the map):
[image: Inline image 1]
I used a printVarsummary(x) for the variable information.
Variable: x
Type: float
Total Size: 11613888 bytes
            2903472 values
Number of Dimensions: 2
Dimensions and sizes: [y | 1872] x [x | 1551]
Coordinates:
Number Of Attributes: 7
  time : 11055.5
  long_name : daily total precipitation
  units : mm/day
  coordinates : lon lat
  _FillValue : -9999
  valid_range : ( 0, 200 )
  cell_methods : area: sum time: sum

and here gave out the error information:
warning:Attempt to reference attribute (date) which is undefined
fatal:ContourPlotDraw: Workspace reallocation would exceed maximum size
32556688
fatal:ContourPlotDraw: draw error
warning:WorkstationDeactivate: workstation not active or not opened
fatal:ContourPlotDraw: Workspace reallocation would exceed maximum size
32556688
fatal:ContourPlotDraw: draw error
warning:WorkstationDeactivate: workstation not active or not opened

How can I fix this(Workspace reallocation exceed maximum size)?

Thanks,

Ping

On Sat, Oct 27, 2012 at 8:05 PM, Dave Allured <dave.allured@noaa.gov> wrote:

> Ping,
>
> Your information helped me find a new problem in the join script.
> When two adjacent tiles are separated by water, the previous version
> would output an invalid merged file with corrupted coordinates, and no
> warning message. This could easily explain your original problem.
>
> I added a consistency check for bad joins with no common grid point,
> and prevented the script from making a bad output file. I have also
> added notes to the web page about Daymet tiles with small data grids.
> Thank you for your help in finding this problem.
>
> Please download and try the latest join script from the web page:
> http://www.esrl.noaa.gov/psd/people/dave.allured/data/daymet/daymet.html
>
> --Dave
>
> On Sat, Oct 27, 2012 at 2:56 PM, Dave Allured <dave.allured@noaa.gov>
> wrote:
> > Ping,
> >
> > I looked at 11393 on the coast. Daymet apparently minimizes the
> > included X/Y grid for each tile to cover points over land only. So
> > the tile collection includes some micro grids. Church Lady says,
> > isn't that special!
> >
> > Well, the join script is capable of joining files of differing grid
> > sizes, because that is the default paradigm for Daymet, anyway. So
> > you are safe to join the small tiles with the normal ones.
> >
> > The primary joining rule remains. Any two files (or "grids") to be
> > joined *by this method* must have at least one grid point in common.
> > So a subsize tile along the coastline should join with the next tile
> > inland, with no problem.
> >
> > Things may break down if you try to join two tiles with no land along
> > the common edge. Now I see this particular problem in your tile map
> > in yesterday's post. The script will probably fail if you try to join
> > 11936 and 11937 directly (east-west). So join 11937 (Yarmouth, Nova
> > Scotia) to land to the north first, to make a "bridge" to the
> > continent. Also, please do inspect the console very carefully for
> > error messages, when joining many tiles under control of a driver
> > script. HTH.
> >
> > --Dave
> >
> > On Fri, Oct 26, 2012 at 8:10 PM, Ping Yang <pyang@ccny.cuny.edu> wrote:
> >> Hi,
> >>
> >> I just ran all the original files, there are several files(size are very
> >> small, especially tiles at the east corner near the atlantic sea) are
> not
> >> 100% percent for every tile, some of them like:
> >> 11393 50%
> >> 11756 20 percent.
> >>
> >> Should I just exclude them(but it will lose the data)?
> >>
> >> Is there a way to deal with those files?
> >>
> >> Looking forward to hearing from you.
> >>
> >> Thank you.
> >>
> >> Regards,
> >>
> >> Ping
> >> On Fri, Oct 26, 2012 at 9:10 PM, Dave Allured <dave.allured@noaa.gov>
> wrote:
> >>>
> >>> One more thing. Would you please run the *other* plot script,
> >>> plot.daymet-orig.1024.ncl, to quickly step through all 68 original
> >>> times and just make sure they all "look" okay. That is, the square
> >>> areas with data should appear to be be 100% present for every tile,
> >>> and adjacent tiles should "look" like they align properly. This is
> >>> not a rigorous test, just a quick visual check. Thank you.
> >>>
> >>> --Dave
> >>>
> >>> On Fri, Oct 26, 2012 at 7:02 PM, Dave Allured <dave.allured@noaa.gov>
> >>> wrote:
> >>> > Hi Ping.
> >>> >
> >>> > Your problem may be in the plot and not the merged data file. Note
> that
> >>> > on
> >>> > my web page I say: "Streaking may occur if any of the missing 2-D
> >>> > coordinates are within the map boundaries". If you made that plot
> with
> >>> > NCL,
> >>> > then you are probably getting severe streaking from the missing 12
> tiles
> >>> > in
> >>> > the southwest corner.
> >>> >
> >>> > An easy check would to be to reduce the map limits in your plotting
> >>> > script,
> >>> > and only plot rectangular areas that are completely covered by Daymet
> >>> > tiles
> >>> > with data.
> >>> >
> >>> > If this is not the problem, then you may need to check very carefully
> >>> > for
> >>> > unexpected gaps in the 2-D coordinates, or similar problems. You may
> >>> > have
> >>> > also found another bug in the join script. You are only the second
> >>> > person
> >>> > to ever try it. ;-)
> >>> >
> >>> > But please first try plotting some map subsets. If you can localize
> >>> > where
> >>> > the problem is in the merged data by this method, that would help the
> >>> > diagnosis.
> >>> >
> >>> > --Dave
> >>> >
> >>> > On Fri, Oct 26, 2012 at 4:59 PM, Ping Yang <pyang@ccny.cuny.edu>
> wrote:
> >>> >>
> >>> >> Dear NCL,
> >>> >>
> >>> >> The Daymet
> >>> >>
> >>> >>
> http://www.esrl.noaa.gov/psd/people/dave.allured/data/daymet/daymet.htmlis
> >>> >> great.
> >>> >>
> >>> >> Thank you so much.
> >>> >>
> >>> >> I got the script for join 68 tiles for the Northeast U.S. using the
> >>> >> attached scripts(a version from the newest ncl code and a bash
> script I
> >>> >> created for the joining)
> >>> >>
> >>> >> However, I found some thing wrong here, I am copying a picture here:
> >>> >>
> >>> >> the tiles that I am working on is:
> >>> >>
> >>> >> I think there is some problem with the joining of with the row 117
> and
> >>> >> row
> >>> >> 119.
> >>> >>
> >>> >> I didn't figure out where is the problem.
> >>> >>
> >>> >> Could you please give me some help.
> >>> >>
> >>> >> Best Regards,
> >>> >>
> >>> >> Ping
>

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk

image.png
Received on Sun Oct 28 19:54:28 2012

This archive was generated by hypermail 2.1.8 : Wed Oct 31 2012 - 09:14:12 MDT