Re: Error when writing global attributes in Netcdf-4 classic file

From: Dave Allured - NOAA Affiliate <dave.allured_at_nyahnyahspammersnyahnyah>
Date: Tue Apr 02 2013 - 12:04:48 MDT

Wei,

Thanks for checking this out. There is definitely a problem
somewhere. Now I am finding the same seg fault on Linux systems as
well as different Mac OS machines, with both NCL 6.0.0 and 6.1.2. In
fact I can not find a local system that does NOT get this seg fault.

Might this be related to supplemental libraries? Following are simple
dependency lists for both Linux and Mac. Let me know if there are any
other diagnostics that I can provide. This is not a high priority for
me, I have a workaround.

Dave

inta:~/temp/3-atts/402a 1045> date
Tue Apr 2 11:49:25 MDT 2013

inta:~/temp/3-atts/402a 1046> uname -a
Linux inta.psd.esrl.noaa.gov 2.6.18-348.3.1.el5 #1 SMP Tue Mar 5
13:19:32 EST 2013 x86_64 GNU/Linux

inta:~/temp/3-atts/402a 1047> echo $NCARG_ROOT
/usr/local/ncl6.1.2

inta:~/temp/3-atts/402a 1048> ls -go $NCARG_ROOT/bin/ncl
-rwxr-xr-x 1 32917555 Feb 6 21:30 /usr/local/ncl6.1.2/bin/ncl

inta:~/temp/3-atts/402a 1049> ls -go
-rw-r--r-- 1 501 Apr 2 10:55 demo.wei-huang.0401a.ncl
-r--r--r-- 1 11294147 Apr 1 17:14 ozone.master.nc

inta:~/temp/3-atts/402a 1050> cat demo.wei-huang.0401a.ncl
begin
  outfile = "out.nc"
; system ("cp ozone.master.nc " + outfile) ; create fresh writable copy
  system ("cat ozone.master.nc > " + outfile) ; create fresh writable copy
  system ("chmod +w " + outfile) ; create fresh writable copy

  out = addfile (outfile, "w")

  out@last_update = systemfunc ("date")
  out@last_update_program = "demo.global-atts.ncl version 2013-apr-01"
  out@last_update_source_file = "ei.ozone.1979010106.nc"
  print ("Update complete.")
 end

inta:~/temp/3-atts/402a 1051> $NCARG_ROOT/bin/ncl demo.wei-huang.0401a.ncl
 Copyright (C) 1995-2013 - All Rights Reserved
 University Corporation for Atmospheric Research
 NCAR Command Language Version 6.1.2
 The use of this software is governed by a License Agreement.
 See http://www.ncl.ucar.edu/ for more details.
(0) Update complete.
Segmentation fault

inta:~/temp/3-atts/402a 1052> ldd $NCARG_ROOT/bin/ncl
        linux-vdso.so.1 => (0x00007fff7dbfd000)
        libcurl.so.3 => /usr/lib64/libcurl.so.3 (0x0000003288400000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003278600000)
        libbz2.so.1 => /usr/lib64/libbz2.so.1 (0x000000328c000000)
        libXrender.so.1 => /usr/lib64/libXrender.so.1 (0x000000327aa00000)
        libX11.so.6 => /usr/lib64/libX11.so.6 (0x0000003278e00000)
        libXext.so.6 => /usr/lib64/libXext.so.6 (0x0000003279a00000)
        libm.so.6 => /lib64/libm.so.6 (0x0000003277a00000)
        libdl.so.2 => /lib64/libdl.so.2 (0x0000003277e00000)
        libstdc++.so.6 => /usr/local/gcc-4.7/lib/libstdc++.so.6 (0x00002b9f6bc6c000)
        libgcc_s.so.1 => /usr/local/gcc-4.7/lib/libgcc_s.so.1 (0x00002b9f6bf76000)
        libc.so.6 => /lib64/libc.so.6 (0x0000003277600000)
        /lib64/ld-linux-x86-64.so.2 (0x0000003277200000)
        libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x0000003282c00000)
        libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x0000003282400000)
        libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x0000003282000000)
        libcom_err.so.2 => /lib64/libcom_err.so.2 (0x0000003281c00000)
        libidn.so.11 => /usr/lib64/libidn.so.11 (0x0000003284000000)
        libssl.so.6 => /lib64/libssl.so.6 (0x0000003288c00000)
        libcrypto.so.6 => /lib64/libcrypto.so.6 (0x0000003288800000)
        libz.so.1 => /lib64/libz.so.1 (0x0000003278200000)
        libXau.so.6 => /usr/lib64/libXau.so.6 (0x0000003279600000)
        libXdmcp.so.6 => /usr/lib64/libXdmcp.so.6 (0x0000003278a00000)
        libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x0000003282800000)
        libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x0000003281400000)
        libresolv.so.2 => /lib64/libresolv.so.2 (0x0000003280c00000)
        libselinux.so.1 => /lib64/libselinux.so.1 (0x000000327d200000)
        libsepol.so.1 => /lib64/libsepol.so.1 (0x000000327ce00000)

========================================================

mac56:~/ncl/bugs/3-atts/402a 35> uname -a
Darwin mac56 12.3.0 Darwin Kernel Version 12.3.0: Sun Jan 6 22:37:10
PST 2013; root:xnu-2050.22.13~1/RELEASE_X86_64 x86_64

mac56:~/ncl/bugs/3-atts/402a 36> echo $NCARG_ROOT
/usr/local/ncl6.1.2

mac56:~/ncl/bugs/3-atts/402a 37> ls -dgo $NCARG_ROOT/bin/ncl
-rwxr-xr-x 1 33112624 Feb 6 21:38 /usr/local/ncl6.1.2/bin/ncl

mac56:~/ncl/bugs/3-atts/402a 38> ldd $NCARG_ROOT/bin/ncl
/usr/local/ncl6.1.2/bin/ncl:
        /usr/lib/libcurl.4.dylib (compatibility version 7.0.0, current version 7.0.0)
        /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
        /usr/X11/lib/libXrender.1.dylib (compatibility version 5.0.0, current
version 5.0.0)
        /usr/lib/libbz2.1.0.dylib (compatibility version 1.0.0, current version 1.0.5)
        /usr/X11/lib/libXpm.4.dylib (compatibility version 16.0.0, current
version 16.0.0)
        /usr/X11/lib/libX11.6.dylib (compatibility version 10.0.0, current
version 10.0.0)
        /usr/X11/lib/libXext.6.dylib (compatibility version 11.0.0, current
version 11.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 159.1.0)
        /usr/local/lib/libstdc++.6.dylib (compatibility version 7.0.0,
current version 7.14.0)
        /usr/local/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current
version 1.0.0)

On Mon, Apr 1, 2013 at 9:35 PM, Wei Huang <huangwei@ucar.edu> wrote:
> Dave,
>
> I can not reproduce your problem.
>
> I only added:
> system ("chmod +w " + outfile)
> after system call in your script (as the original file is write protected).
>
> and it runs fine.
>
> I tested on Mac, with osx 10.8.3.
>
> Here is the full script I tested:
>
> ; 2013-apr-01 Test program reduced from era-update.ncl.
> ; Demonstrates seg fault when trying to overwrite global
> ; attributes in a Netcdf-4 classic file.
> ; By Dave Allured, NOAA/PSD/CIRES Climate Analysis Branch.
>
> begin
> outfile = "out.nc"
> ; system ("cp ozone.master.nc " + outfile) ; create fresh writable copy
> system ("cat ozone.master.nc > " + outfile) ; create fresh writable copy
> system ("chmod +w " + outfile) ; create fresh writable copy
>
> out = addfile (outfile, "w")
>
> out@last_update = systemfunc ("date")
> out@last_update_program = "demo.global-atts.ncl version 2013-apr-01"
> out@last_update_source_file = "ei.ozone.1979010106.nc"
> print ("Update complete.")
> end
>
>
> and part of ncdump -h out.nc:
>
> // global attributes:
> :title = "ERA-Interim Ozone mass mixing ratio, 4 times daily" ;
> :history = "Mon Apr 1 17:14:49 MDT 2013:\n",
> "Initial file made by era-create.ncl version 2013-apr-01 TEST" ;
> :dataset = "ERA-Interim" ;
> :version = "2.0" ;
> :references = "http://www.ecmwf.int/research/era/do/get/era-interim;\n",
> "D. P. Dee et al (2011), The ERA-Interim reanalysis:\n",
> " configuration and performance of the data assimilation system,\n",
> " Q.J.R. Meteorol. Soc., Vol. 137: 553-597, DOI: 10.1002/qj.828" ;
> :center = "European Center for Medium-Range Weather Forecasts (RSMC)" ;
> :metadata_source_file = "../ei.oper.an.pl.regn128sc.1979010100" ;
> :Conventions = "CF-1.6" ;
> :last_update = "Mon Apr 1 21:28:20 MDT 2013" ;
> :last_update_program = "demo.global-atts.ncl version 2013-apr-01" ;
> :last_update_source_file = "ei.ozone.1979010106.nc" ;
> }
>
>
> Wei
>
> huangwei@ucar.edu
> VETS/CISL
> National Center for Atmospheric Research
> P.O. Box 3000 (1850 Table Mesa Dr.)
> Boulder, CO 80307-3000 USA
> (303) 497-8924
>
>
>
>
>
>
> On Apr 1, 2013, at 8:41 PM, Dave Allured - NOAA Affiliate <dave.allured@noaa.gov> wrote:
>
>> NCL support team,
>>
>> This program results in a seg fault error on program exit, after
>> attempting to overwrite some global attributes in a Netcdf-4 classic
>> file with HDF5 compression. Note that all three global attributes
>> exist previously in the target file, before running the test program.
>>
>> After the error, the target file is corrupted. This error is
>> strangely sensitive to the number and string lengths of the attributes
>> to be written. Can you please tell me what the problem is? Thank
>> you.
>>
>> begin
>> outfile = "out.nc"
>> system ("cat ozone.master.nc > " + outfile) ; create fresh writable copy
>>
>> out = addfile (outfile, "w")
>> out@last_update = systemfunc ("date")
>> out@last_update_program = "demo.global-atts.ncl version 2013-apr-01"
>> out@last_update_source_file = "ei.ozone.1979010106.nc"
>> print ("Update complete.")
>> end
>>
>> I get the same results with NCL versions 6.0.0 and 6.1.2. My system
>> is Mac OS 10.8.3 on a 64-bit Intel Mac desktop. Here is the console
>> output:
>>
>> mac56:~/ncl/bugs/3-atts/401i 491> ncl demo.global-atts.ncl
>> Copyright (C) 1995-2013 - All Rights Reserved
>> University Corporation for Atmospheric Research
>> NCAR Command Language Version 6.1.2
>> The use of this software is governed by a License Agreement.
>> See http://www.ncl.ucar.edu/ for more details.
>> (0) Update complete.
>> Segmentation fault
>>
>> mac56:~/ncl/bugs/3-atts/401i 492> ls -trgo
>> total 44136
>> -r--r--r-- 1 11294147 Apr 1 17:14 ozone.master.nc
>> -rw-r--r-- 1 570 Apr 1 19:51 demo.global-atts.ncl
>> -rw-r--r-- 1 11294201 Apr 1 20:16 out.nc
>>
>> mac56:~/ncl/bugs/3-atts/401i 493> ncdump -k ozone.master.nc
>> netCDF-4 classic model
>>
>> mac56:~/ncl/bugs/3-atts/401i 494> ncdump -k out.nc
>> ncdump: out.nc: NetCDF: Can't open HDF5 attribute
>>
>> mac56:~/ncl/bugs/3-atts/401i 495> uname -a
>> Darwin mac56 12.3.0 Darwin Kernel Version 12.3.0: Sun Jan 6 22:37:10
>> PST 2013; root:xnu-2050.22.13~1/RELEASE_X86_64 x86_64
>>
>> I uploaded the program and original Netcdf file (11 Mbytes) to
>> ftp.cgd.ucar.edu/incoming, as
>> "demo.global-atts.dave_allured.0401.zip". The original Netcdf file
>> was also made by NCL 6.1.2 and a different script. I can also send a
>> copy of the generating script and its input file, if you need it.
>> Please let me know. Thanks for taking a look at this.
>>
>> --Dave A.
>> NOAA/PSD/CIRES Climate Analysis Branch
_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Tue Apr 2 12:05:04 2013

This archive was generated by hypermail 2.1.8 : Tue Apr 02 2013 - 21:23:48 MDT