Re: DEM plot: memory allocation error

Rick is correct about the resolution issue, and I would like to add a couple of more points.
1) You are plotting this as a raster fill plot, which means the pixel resolution of the image is by default limited to 1024 x 1024, so without setting other resources to increase the resolution you would
be trying to put the information from 27260 * 26610 / 1024^2 (= ~ 692) data locations into each pixel. The resulting plot would effectively not give any more information than if you stride the data quite a bit more aggressively than Rick suggests.
2) Even though NCL variables can now be bigger than 2 GB on 64 bit systems, the graphics code still has a 2GB limit for data that can be plotted, so at least some striding is required to plot this dataset.
3) There is a way to save some dynamic memory if you adhere to certain restrictions. Set the resource sfCopyData to False. Here is the doc for this resource:

The boolean resource sfCopyData allows you to control whether the ScalarField object always copies the elements of the sfDataArray into a private memory area. If you set sfCopyData False, ScalarField will not copy the sfXArray or the sfYArray data and will try to avoid making a copy of the sfDataArray data. You are then responsible for ensuring that the data memory locations for these arrays are preserved intact for the lifetime of the ScalarField. In return, you save some dynamic memory space. Note that a number of circumstances require ScalarField to copy the data regardless. These include supplying data of any type other than NhlTFloat, requesting that the data dimensions be exchanged, or setting a stride value greater than 1 in either the X or the Y dimension. Also, if you specify a data array subset and one or both of the coordinate array resources (sfXArray and/or sfYArray) is non-NULL, copies are made of the subsetted part of the coordinate array.
Default: True

Note that this resource has not been used or advertised much. We think it works but it has not been tested heavily.
 -dave

On Apr 11, 2013, at 9:16 AM, Rick Brownrigg <brownrig@ucar.edu> wrote:

> Hi Noel,
>
> Memory issues aside, it doesn't seem like you'll be able to graphically depict both the full areal extent of that data (1500miles wide?) and resolve its full detail. What if you were to resample it at a lower resolution; perhaps something as simple as:
>
> plot = gsn_csm_contour_map(wks, dem90m(::2,::2), res1)
>
> Just a thought…
>
> Rick
>
> On Apr 11, 2013, at 9:01 AM, Noel Aloysius <noel.aloysius@gmail.com> wrote:
>
>> Hi Mary,
>>
>> The variable type is float. Should I convert the type to integer and work.
>>
>> Noel
>>
>> Noel Aloysius
>>
>>
>> On Thu, Apr 11, 2013 at 10:53 AM, Mary Haley <haley@ucar.edu> wrote:
>> Hi Noel,
>>
>> Sorry, I've been away from ncl-talk for a bit.
>>
>> This is not something you can fix with wsMaximumSize, unfortunately. I think internally, NCL might be making a copy of your array, and simply running out of memory.
>>
>> What type is the variable?
>>
>> I'm hoping Dave Brown will have an idea for a work-around, that might prevent a copy from taking place.
>>
>> --Mary
>>
>>
>> On Apr 8, 2013, at 1:17 PM, Noel Aloysius wrote:
>>
>> > Dear NCL-talk,
>> >
>> > I have a 90m DEM with the following dimensions (/27260,26610/), ~ 2.9GB in size. The script below creates a spatial plot of the DEM.
>> >
>> > The plot is successfully created if I restrict the region as follows,
>> >
>> > plot = gsn_csm_contour_map(wks, dem90m(5000:27259,5000:26609), res1)
>> >
>> > I get an error otherwise.
>> >
>> > fatal:NhlRealloc Failed:[errno=12]
>> > fatal:ContourPlotDraw: dynamic memory allocation error
>> > fatal:ContourPlotDraw: draw error
>> > warning:WorkstationDeactivate: workstation not active or not opened
>> >
>> > Is there a way around?
>> >
>> > Thanks,
>> > Noel
>> >
>> >
>> > ;*****************************************************************
>> > ffig = diro + "dem90m"
>> > wks = gsn_open_wks(type, ffig)
>> > plot = new(1, graphic)
>> > gsn_define_colormap(wks, "ncl_default")
>> > maps = NewList("fifo")
>> >
>> > ;--------------------------------------------------------
>> > setvalues NhlGetWorkspaceObjectId()
>> > "wsMaximumSize": 2000000000
>> > end setvalues
>> > ;--------------------------------------------------------
>> >
>> > res1 = True ; plot mods desired
>> > res1@gsnDraw = False
>> > res1@gsnFrame = False
>> >
>> > res1@tiMainString = "90m DEM"
>> > res1@gsnPaperOrientation = "auto"
>> > res1@gsnMaximize = True
>> > res1@gsnSpreadColors = True
>> > res1@gsnAddCyclic = False
>> >
>> > res1@lbLabelBarOn = True
>> >
>> > res1@cnFillMode = "RasterFill"
>> > res1@cnFillOn = True
>> > res1@cnLinesOn = False
>> >
>> > res1@mpCenterLonF = (regBnds(2)+regBnds(3))/2 ; center the map arounf this lon.
>> > res1@mpMinLatF = regBnds(0) ; select a subregion
>> > res1@mpMaxLatF = regBnds(1)
>> > res1@mpMinLonF = regBnds(2)
>> > res1@mpMaxLonF = regBnds(3)
>> >
>> > plot = gsn_csm_contour_map(wks, dem90m(5000:27259,5000:26609), res1)
>> > ;plot = gsn_csm_contour_map(wks, dem90m, res1)
>> >
>> > ; draw polygon
>> > lnres1 = True
>> > lnres1@gsLineColor = "black"
>> > lnres1@gsLineThicknessF = 2.0
>> > ListPush(maps, gsn_add_shapefile_polylines(wks,plot,fshp1,lnres1))
>> >
>> > draw(wks)
>> > frame(wks)
>> > ;***************************************************************
>> >
>> > _______________________________________________
>> > ncl-talk mailing list
>> > List instructions, subscriber options, unsubscribe:
>> > http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>
>>
>> _______________________________________________
>> ncl-talk mailing list
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Thu Apr 11 11:26:02 2013