Re: Seg Fault when Subsetting Large Files

From: Carl Schreck <cjschrec_at_nyahnyahspammersnyahnyah>
Date: Mon Mar 03 2014 - 07:30:19 MST

Hi Rick,

Here are the details of my machine:
$ uname -a
Linux c140.cluster 2.6.32-431.3.1.el6.x86_64 #1 SMP Fri Dec 13 06:58:20 EST
2013 x86_64 x86_64 x86_64 GNU/Linux

$ gcc --version
gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-4)

And I am using the NCL from:
ncl_ncarg-6.1.2.Linux_RHEL6.2_x86_64_gcc446.tar.gz

And here's the output from the seg fault. I'm not familiar with gdb, so let
me know if this isn't what you were looking for:

Program received signal SIGSEGV, Segmentation fault.
0x0000000000c83f4e in swapn4b ()
Missing separate debuginfos, use: debuginfo-install
bzip2-libs-1.0.5-7.el6_0.x86_64 cyrus-sasl-lib-2.1.23-13.el6_3.1.x86_64
glibc-2.12-1.132.el6.x86_64 keyutils-libs-1.4-4.el6.x86_64
krb5-libs-1.10.3-10.el6_4.6.x86_64 libcom_err-1.41.12-18.el6.x86_64
libcurl-7.19.7-37.el6_4.x86_64 libgcc-4.4.7-4.el6.x86_64
libidn-1.18-2.el6.x86_64 libselinux-2.0.94-5.3.el6_4.1.x86_64
libssh2-1.4.2-1.el6.x86_64 libstdc++-4.4.7-4.el6.x86_64
nspr-4.10.2-1.el6_5.x86_64 nss-3.15.3-3.el6_5.x86_64
nss-pam-ldapd-0.7.5-18.2.el6_4.x86_64
nss-softokn-freebl-3.14.3-9.el6.x86_64 nss-util-3.15.3-1.el6_5.x86_64
openldap-2.4.23-32.el6_4.1.x86_64 openssl-1.0.1e-16.el6_5.4.x86_64
zlib-1.2.3-29.el6.x86_64
(gdb) bt
#0 0x0000000000c83f4e in swapn4b ()
#1 0x0000000000c890c4 in ncx_getn_float_float ()
#2 0x0000000000c93c5d in getNCvx_float_float ()
#3 0x0000000000c95c8f in readNCv ()
#4 0x0000000000c96b58 in NC3_get_vara ()
#5 0x0000000000c7ab78 in NC_get_vara ()
#6 0x0000000000c7b7fe in nc_get_vara ()
#7 0x0000000000c74f72 in ncvarget ()
#8 0x0000000000c750bb in ncvargets ()
#9 0x0000000000c75258 in ncvargetg ()
#10 0x00000000006aa712 in NetReadVar ()
#11 0x0000000000689f79 in MyFileReadVarValue ()
#12 0x000000000068c22b in FileReadVar ()
#13 0x0000000000663724 in _NclFileReadVar ()
#14 0x00000000007221ed in CallFILE_VAR_OP ()
#15 0x00000000007306fa in _NclExecute ()
#16 0x0000000000614410 in yyparse ()
#17 0x000000000060f8e7 in main ()

Thanks!

    Carl

On Fri, Feb 28, 2014 at 4:37 PM, Rick Brownrigg <brownrig@ucar.edu> wrote:

> Hi Carl,
>
> What platform and ncl-version are you running on? Can you run ncl under
> gdb with this script and when it SEGV's, do a "bt" command?
>
> Curiously, the script below works fine on my 8GB Mac. (It failed
> gracefully on my 4GB linux machine with a "can't allocate 3.7GB" error
> message).
>
> Rick
>
> On Feb 28, 2014, at 12:28 PM, Carl Schreck <cjschrec@ncsu.edu> wrote:
>
> I'm getting a seg fault when I try to subset a large file (5 GB). Consider
> this small example:
>
> path = "/snfs0/scratch/carl/u850.anom.nc"
> inFile = addfile( path, "r" )
>
> x0 = 5
> xn = 200
>
> data1 = inFile->u850(:,:,x0:xn)
> printVarSummary(data1)
> delete(data1)
>
> xInd = ispan(x0,xn,1)
> data2 = inFile->u850(:,:,xInd)
> printVarSummary(data2)
>
> It works fine on the first attempt when I read it using a range x0:xn.
> However, it seg faults on the second attempt where I try to subset it by an
> array (xInd). The machine I'm on has 120 GB of RAM, so it shouldn't be a
> memory issue.
>
> You can download the file at:
> ftp://filsrv.cicsnc.org/carl/largeFile/
> I also made the file in NCL, so I'll be curious if it's something that's
> improperly formatted there.
>
> Thanks for your help!
>
> Carl
>
> --
>
> <http://www.cicsnc.org/>
> <http://www.facebook.com/cicsnc>
> *Carl J. Schreck III, PhD*
> *Research Associate*
> Cooperative Institute for Climate and Satellites NC <http://cicsnc.org/>
> North Carolina State University <http://ncsu.edu/>
> NOAA's National Climatic Data Center <http://ncdc.noaa.gov/>
> 151 Patton Ave, Asheville, NC 28801
> e: cjschrec@ncsu.edu
> o: +1 828 257 3140
> CycloneCenter.org: Help analyze tropical cyclones<http://www.cyclonecenter.org/>
> monitor.cicsnc.org/mjo: See recent MJO diagnostics<http://monitor.cicsnc.org/mjo/>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
>

-- 
 <http://www.cicsnc.org/>
 <http://www.facebook.com/cicsnc>
*Carl J. Schreck III, PhD*
*Research Associate*
Cooperative Institute for Climate and Satellites NC <http://cicsnc.org/>
North Carolina State University <http://ncsu.edu/>
NOAA's National Climatic Data Center <http://ncdc.noaa.gov/>
151 Patton Ave, Asheville, NC 28801
e: cjschrec@ncsu.edu
o: +1 828 257 3140
CycloneCenter.org: Help analyze tropical cyclones<http://www.cyclonecenter.org/>
monitor.cicsnc.org/mjo: See recent MJO
diagnostics<http://monitor.cicsnc.org/mjo/>

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Mon Mar 3 07:31:18 2014

This archive was generated by hypermail 2.1.8 : Mon Mar 03 2014 - 14:26:18 MST