Re: Seg Fault when Subsetting Large Files

Hi Carl,

I can reproduce the SEGV under Ncl 6.1.2 on every machine I can put my hands on, but the script runs fine under the upcoming 6.2.0. Without going into details (because I don't understand them myself!), my colleagues tell me that there were some changes to the NetCDf reader between 6.1.2 and 6.2.0, and that these likely are at the root of the problem/solution. I will see about getting you a pre 6.2.0 binary, which should get you past the problem.

Rick

On Mar 3, 2014, at 7:30 AM, Carl Schreck <cjschrec@ncsu.edu> wrote:

> Hi Rick,
>
> Here are the details of my machine:
> $ uname -a
> Linux c140.cluster 2.6.32-431.3.1.el6.x86_64 #1 SMP Fri Dec 13 06:58:20 EST 2013 x86_64 x86_64 x86_64 GNU/Linux
>
> $ gcc --version
> gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-4)
>
> And I am using the NCL from:
> ncl_ncarg-6.1.2.Linux_RHEL6.2_x86_64_gcc446.tar.gz
>
> And here's the output from the seg fault. I'm not familiar with gdb, so let me know if this isn't what you were looking for:
>
> Program received signal SIGSEGV, Segmentation fault.
> 0x0000000000c83f4e in swapn4b ()
> Missing separate debuginfos, use: debuginfo-install bzip2-libs-1.0.5-7.el6_0.x86_64 cyrus-sasl-lib-2.1.23-13.el6_3.1.x86_64 glibc-2.12-1.132.el6.x86_64 keyutils-libs-1.4-4.el6.x86_64 krb5-libs-1.10.3-10.el6_4.6.x86_64 libcom_err-1.41.12-18.el6.x86_64 libcurl-7.19.7-37.el6_4.x86_64 libgcc-4.4.7-4.el6.x86_64 libidn-1.18-2.el6.x86_64 libselinux-2.0.94-5.3.el6_4.1.x86_64 libssh2-1.4.2-1.el6.x86_64 libstdc++-4.4.7-4.el6.x86_64 nspr-4.10.2-1.el6_5.x86_64 nss-3.15.3-3.el6_5.x86_64 nss-pam-ldapd-0.7.5-18.2.el6_4.x86_64 nss-softokn-freebl-3.14.3-9.el6.x86_64 nss-util-3.15.3-1.el6_5.x86_64 openldap-2.4.23-32.el6_4.1.x86_64 openssl-1.0.1e-16.el6_5.4.x86_64 zlib-1.2.3-29.el6.x86_64
> (gdb) bt
> #0 0x0000000000c83f4e in swapn4b ()
> #1 0x0000000000c890c4 in ncx_getn_float_float ()
> #2 0x0000000000c93c5d in getNCvx_float_float ()
> #3 0x0000000000c95c8f in readNCv ()
> #4 0x0000000000c96b58 in NC3_get_vara ()
> #5 0x0000000000c7ab78 in NC_get_vara ()
> #6 0x0000000000c7b7fe in nc_get_vara ()
> #7 0x0000000000c74f72 in ncvarget ()
> #8 0x0000000000c750bb in ncvargets ()
> #9 0x0000000000c75258 in ncvargetg ()
> #10 0x00000000006aa712 in NetReadVar ()
> #11 0x0000000000689f79 in MyFileReadVarValue ()
> #12 0x000000000068c22b in FileReadVar ()
> #13 0x0000000000663724 in _NclFileReadVar ()
> #14 0x00000000007221ed in CallFILE_VAR_OP ()
> #15 0x00000000007306fa in _NclExecute ()
> #16 0x0000000000614410 in yyparse ()
> #17 0x000000000060f8e7 in main ()
>
> Thanks!
>
> Carl
>
>
> On Fri, Feb 28, 2014 at 4:37 PM, Rick Brownrigg <brownrig@ucar.edu> wrote:
> Hi Carl,
>
> What platform and ncl-version are you running on? Can you run ncl under gdb with this script and when it SEGV's, do a "bt" command?
>
> Curiously, the script below works fine on my 8GB Mac. (It failed gracefully on my 4GB linux machine with a "can't allocate 3.7GB" error message).
>
> Rick
>
> On Feb 28, 2014, at 12:28 PM, Carl Schreck <cjschrec@ncsu.edu> wrote:
>
>> I'm getting a seg fault when I try to subset a large file (5 GB). Consider this small example:
>>
>> path = "/snfs0/scratch/carl/u850.anom.nc"
>> inFile = addfile( path, "r" )
>>
>> x0 = 5
>> xn = 200
>>
>> data1 = inFile->u850(:,:,x0:xn)
>> printVarSummary(data1)
>> delete(data1)
>>
>> xInd = ispan(x0,xn,1)
>> data2 = inFile->u850(:,:,xInd)
>> printVarSummary(data2)
>>
>> It works fine on the first attempt when I read it using a range x0:xn. However, it seg faults on the second attempt where I try to subset it by an array (xInd). The machine I'm on has 120 GB of RAM, so it shouldn't be a memory issue.
>>
>> You can download the file at:
>> ftp://filsrv.cicsnc.org/carl/largeFile/
>> I also made the file in NCL, so I'll be curious if it's something that's improperly formatted there.
>>
>> Thanks for your help!
>>
>> Carl
>>
>> --
>>
>>
>>
>> Carl J. Schreck III, PhD
>> Research Associate
>> Cooperative Institute for Climate and Satellites NC
>> North Carolina State University
>> NOAA's National Climatic Data Center
>> 151 Patton Ave, Asheville, NC 28801
>> e: cjschrec@ncsu.edu
>> o: +1 828 257 3140
>> CycloneCenter.org: Help analyze tropical cyclones
>> monitor.cicsnc.org/mjo: See recent MJO diagnostics
>> _______________________________________________
>> ncl-talk mailing list
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
>
>
> --
>
>
>
> Carl J. Schreck III, PhD
> Research Associate
> Cooperative Institute for Climate and Satellites NC
> North Carolina State University
> NOAA's National Climatic Data Center
> 151 Patton Ave, Asheville, NC 28801
> e: cjschrec@ncsu.edu
> o: +1 828 257 3140
> CycloneCenter.org: Help analyze tropical cyclones
> monitor.cicsnc.org/mjo: See recent MJO diagnostics

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Mon Mar 3 14:31:30 2014