Re: Segmentation Fault

Carl and Mike,

After some debugging, we have figured out that:
1. Your dataset is chunked, compressed data:

(0) file compression level = 1
(0) getfilechunksizes = 1
(1) getfilechunksizes = 121
(2) getfilechunksizes = 240

2. The segmentation fault problem is from NetCDF library.
        We have a fix to the NetCDF library, and reported it
        to the NetCDF group.

Since this not a direct problem in NCL, so fix the problem need more work.
We probably need to wait NetCDF to conform that our fix to their code is
proper, and then we can update our NCL package.

As you can read the whole data-set, with operation:

x = f->vp,

you may try:

y = x(:,0,:)

To save memory, you can do:

delete(x)

(to remove x from memory)

Please let us know if there is anything we can help.

Wei

huangwei@ucar.edu
VETS/CISL
National Center for Atmospheric Research
P.O. Box 3000 (1850 Table Mesa Dr.)
Boulder, CO 80307-3000 USA
(303) 497-8924

On Feb 1, 2012, at 1:47 PM, David Brown wrote:

> Hi Carl,
> I am looking into this. So far all I can tell is that the seg fault happens within a call to the NetCDF library. i am trying to probe it further to see where the fault really lies.
> -dave
>
> On Feb 1, 2012, at 6:47 AM, Carl Schreck wrote:
>
>> Hey guys,
>>
>> I'm also trying to help Mike with this problem. The results are pretty similar using 6.0.0:
>>
>> ----
>> $ ncl
>> Copyright (C) 1995-2011 - All Rights Reserved
>> University Corporation for Atmospheric Research
>> NCAR Command Language Version 6.0.0
>> The use of this software is governed by a License Agreement.
>> See http://www.ncl.ucar.edu/ for more details.
>> ncl 0> f=addfile("vp.850.nc","r")
>> ncl 1> x=f->vp
>> ncl 2> printVarSummary(x)
>>
>> Variable: x
>> Type: float
>> Total Size: 3873819840 bytes
>> 968454960 values
>> Number of Dimensions: 3
>> Dimensions and sizes: [time | 33349] x [lat | 121] x [lon | 240]
>> Coordinates:
>> time: [1656744..1856832]
>> lat: [-90..90]
>> lon: [-180..178.5]
>> Number Of Attributes: 6
>> _FillValue : -999.9
>> mapping : cylindrical_equidistant_projection_grid
>> grid_resolution : 1.5_degrees
>> pressure_level : 850
>> units : m^2s^-1
>> long_name : velocity_potential
>> ncl 3> y=f->vp(:,0,:)
>> Segmentation fault
>>
>> ----
>>
>> So on line 1, it successfully reads the entire variable at once. But in line 3, it seg faults when we try to read a single latitude (or even a range like 0:9). This is using ncl_ncarg-6.0.0.Linux_RedHat_x86_64_gcc412.tar.gz
>>
>> I'm sure Mike can put the data somewhere that you can access it in case it's a problem with the actual file.
>>
>> Carl
>>
>>
>> On Tue, Jan 31, 2012 at 3:58 PM, Ventrice, Michael J <mventrice@albany.edu> wrote:
>> Dennis and Rick,
>>
>> Yes, ok do I understand that. But why am I still seg-faulting in 6.0.0 when I read in the first latitude and not all latidudes?
>>
>> Thanks,
>> Mike
>>
>>
>> --
>> Michael J. Ventrice
>> Department of Atmospheric and Environmental Sciences
>> University at Albany, New York, ES 218
>> MVentrice@albany.edu
>> Mike.Ventrice@gmail.com
>> http://mikeventrice.weebly.com/index.html
>>
>> ________________________________________
>> From: Dennis Shea [shea@ucar.edu]
>> Sent: Tuesday, January 31, 2012 3:49 PM
>> To: Ventrice, Michael J
>> Cc: Rick Brownrigg; ncl-talk@ucar.edu
>> Subject: Re: Segmentation Fault
>>
>> NCL v5.2.1 does not support variables > 2GB
>>
>> The variable is 33349*121*240 * 4 > 2GB
>>
>> The 4 is because each float is 4 bytes
>>
>>
>> On 1/31/12 1:09 PM, Rick Brownrigg wrote:
>>
>> Variable: x
>> Type: float
>> Total Size: -421147456 bytes
>> 968454960 values
>> Number of Dimensions: 3
>> Dimensions and sizes: [time | 33349] x [lat | 121] x [lon | 240]
>>
>> You must upgrade to v6.0.0 for variables > 2GB
>>
>>
>> > Hi Mike,
>> >
>> > The output you show is from ncl5.2.1, and it certainly makes sense
>> > (e.g., the Total size field reporting "-421147456 bytes"
>> > Can you post the results of the same commands entered into V6.0.0 ?'
>> > What do you mean by "read the data in as a whole"?
>> >
>> > Thanks,
>> > Rick
>> >
>> > On Jan 31, 2012, at 12:27 PM, Ventrice, Michael J wrote:
>> >
>> >> Hello,
>> >>
>> >> I am working the 1.5 deg ECMWF interim data where I have came across a
>> >> problem I am unable to fix. I have built large-files consisting of Jan
>> >> 1989 - September 2011 data, and am able to read the data in as a
>> >> whole, but when I try to read the first latitude point in, I recieve a
>> >> segmentation fault. I recieve segmantation faults for both v5.2.1 and
>> >> v.6.0 of NCL. Would you be able to attempt to help me understand what
>> >> is going wrong?
>> >>
>> >> Thank you,
>> >> Mike Ventrice
>> >>
>> >> ncl
>> >> Copyright (C) 1995-2010 - All Rights Reserved
>> >> University Corporation for Atmospheric Research
>> >> NCAR Command Language Version 5.2.1
>> >> The use of this software is governed by a License Agreement.
>> >> Seehttp://www.ncl.ucar.edu/for more details.
>> >> ncl 0> f=addfile("vp.850.nc <http://vp.850.nc>","r")
>> >> ncl 1> x=f->vp
>> >> ncl 2> printVarSummary(x)
>> >>
>> >>
>> >> Variable: x
>> >> Type: float
>> >> Total Size: -421147456 bytes
>> >> 968454960 values
>> >> Number of Dimensions: 3
>> >> Dimensions and sizes: [time | 33349] x [lat | 121] x [lon | 240]
>> >> Coordinates:
>> >> time: [1656744..1856832]
>> >> lat: [-90..90]
>> >> lon: [-180..178.5]
>> >> Number Of Attributes: 6
>> >> _FillValue : -999.9
>> >> mapping : cylindrical_equidistant_projection_grid
>> >> grid_resolution : 1.5_degrees
>> >> pressure_level : 850
>> >> units : m^2s^-1
>> >> long_name : velocity_potential
>> >> ncl 3> delete(x)
>> >> ncl 4> y=f->vp(:,0,:)
>> >> Segmentation fault
>> >>
>> >> --
>> >> Michael J. Ventrice
>> >> Department of Atmospheric and Environmental Sciences
>> >> University at Albany, New York, ES 218
>> >> MVentrice@albany.edu <mailto:MVentrice@albany.edu>
>> >> Mike.Ventrice@gmail.com <mailto:Mike.Ventrice@gmail.com>
>> >> http://mikeventrice.weebly.com/index.html
>> >> _______________________________________________
>> >> ncl-talk mailing list
>> >> List instructions, subscriber options, unsubscribe:
>> >> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>> >
>> >
>> >
>> > _______________________________________________
>> > ncl-talk mailing list
>> > List instructions, subscriber options, unsubscribe:
>> > http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>> _______________________________________________
>> ncl-talk mailing list
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>
>>
>>
>> --
>> Carl J. Schreck III, PhD
>> Postdoctoral Research Associate
>> Cooperative Institute for Climate and Satellites (CICS-NC)
>> North Carolina State University and NOAA's National Climatic Data Center
>> 151 Patton Avenue
>> Asheville, NC 28801
>> Tel: 828-257-3140
>> carl@cicsnc.org
>> http://monitor.cicsnc.org/mjo/
>> http://www.atmos.albany.edu/student/carl/
>>
>> _______________________________________________
>> ncl-talk mailing list
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Thu Feb 2 03:13:11 2012