Wei,
Thanks for the clarification. It's helpful.
Below lists my observations, just for your information.
a = addfile("a.nc","r")
setfileoption("nc", "Format", "NetCDF4")
c = addfile("ad.nc","c")
; copy variables
; this does not work -- c->/Times = a->/Times
;fatal:["NclNetCDF4.c":5211]:NetCDF: Name contains illegal characters
;fatal:["NclAdvancedFile.c":5640]:AdvancedFileAddVar: Error adding
;variable </Times> as type: <none>
; this works -- c->Times = a->/Times
names = getfilevarnames(a)
do i=0, dimsizes(names)-1
; this does not work -- str_sub_str(names(i), "/", "")
; this works -- replaceSingleChar(names(i), "/", "")
c->$names(i)$ = a->$names(i)$
end do
Now a new question is how to use str_sub_str to get rid of the /
character?
Hui-Chuan
On 4/2/13 3:49 PM, Wei Huang wrote:
> Hui-Chun,
>
> This is kind of a feature for NetCDF4 data.
>
> In NetCDF4, people can use the concept of group(s),
> which means, in a file, in can have group(s) under group(s).
> Which is pretty like our file-system, where we have
> directory(ies) and sub-directory(ies), and certainly files.
>
> So, we may have a variable T in group_A, and
> another variable also named T in group_B.
> Their (relative) names are the same, but they may have
> different dimensions, or could be two totally different things.
>
> There will be a problem, when we have:
>
> f = addfile("nc4_filename.nc", "r")
> t = f->T
>
> which there is ambiguity here of reading which T.
>
> To avoid this issue, user can read T from group_A by using:
>
> t = f->/group_A/T
>
> with this syntax, read T from root-group becomes:
>
> t = f->/T
>
> The problem here is that for lots of users, which they do not have
> any group in their files, and this "extra" "/" cause some confusion.
> BUT this "/" only appears for NetCDF4 files (and will appear
> for HDF5 in 6.2.0, and probably HDF-EOS5 in 6.2.0).
>
> To make users code code compatible with their early code,
> t = f->T will still work.
>
> But users with NetCDF4 files, which has more than one variables
> named "T", the above code will pick the first variable named "T".
>
> Hope this makes things clearly.
>
> Thanks,
>
> Wei
>
>
> huangwei@ucar.edu
> VETS/CISL
> National Center for Atmospheric Research
> P.O. Box 3000 (1850 Table Mesa Dr.)
> Boulder, CO 80307-3000 USA
> (303) 497-8924
>
>
>
>
>
>
> On Apr 2, 2013, at 2:58 PM, Hui-Chuan Lin <hclin@ucar.edu> wrote:
>
>> I've seen similar seg fault problems back in Feb on yellowstone
>> with compressed netcdf-4 files. For example, addfile with "w"
>> and getvaratts() didn't work properly. I came up with workarounds
>> to do what I needed to do.
>> After reading Dave's email, I re-did some tests.
>> Now with 6.1.2, addfile with "w" and getvaratts() seem working,
>> but I encountered a new problem with getfilevarnames().
>> There is an extra / in the beginning of the variable name.
>>
>> A = addfile("netcdf4file.nc", "r")
>> varnames = getfilevarnames(A)
>> print(varnames)
>>
>> Variable: varnames
>> Type: string
>> Total Size: 1112 bytes
>> 139 values
>> Number of Dimensions: 1
>> Dimensions and sizes: [139]
>> Coordinates:
>> (0) /Times
>> (1) /LU_INDEX
>> (2) /ZNU
>> (3) /ZNW
>>
>> Hui-Chuan
>>
>> On 4/2/13 1:28 PM, Wei Huang wrote:
>>> For tracking purpose, I have created a JIRA ticket 1692.
>>>
>>> Wei
>>>
>>> huangwei@ucar.edu
>>> VETS/CISL
>>> National Center for Atmospheric Research
>>> P.O. Box 3000 (1850 Table Mesa Dr.)
>>> Boulder, CO 80307-3000 USA
>>> (303) 497-8924
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Apr 2, 2013, at 12:04 PM, Dave Allured - NOAA Affiliate <dave.allured@noaa.gov> wrote:
>>>
>>>> Wei,
>>>>
>>>> Thanks for checking this out. There is definitely a problem
>>>> somewhere. Now I am finding the same seg fault on Linux systems as
>>>> well as different Mac OS machines, with both NCL 6.0.0 and 6.1.2. In
>>>> fact I can not find a local system that does NOT get this seg fault.
>>>>
>>>> Might this be related to supplemental libraries? Following are simple
>>>> dependency lists for both Linux and Mac. Let me know if there are any
>>>> other diagnostics that I can provide. This is not a high priority for
>>>> me, I have a workaround.
>>>>
>>>> Dave
>>>>
>>>> inta:~/temp/3-atts/402a 1045> date
>>>> Tue Apr 2 11:49:25 MDT 2013
>>>>
>>>> inta:~/temp/3-atts/402a 1046> uname -a
>>>> Linux inta.psd.esrl.noaa.gov 2.6.18-348.3.1.el5 #1 SMP Tue Mar 5
>>>> 13:19:32 EST 2013 x86_64 GNU/Linux
>>>>
>>>> inta:~/temp/3-atts/402a 1047> echo $NCARG_ROOT
>>>> /usr/local/ncl6.1.2
>>>>
>>>> inta:~/temp/3-atts/402a 1048> ls -go $NCARG_ROOT/bin/ncl
>>>> -rwxr-xr-x 1 32917555 Feb 6 21:30 /usr/local/ncl6.1.2/bin/ncl
>>>>
>>>> inta:~/temp/3-atts/402a 1049> ls -go
>>>> -rw-r--r-- 1 501 Apr 2 10:55 demo.wei-huang.0401a.ncl
>>>> -r--r--r-- 1 11294147 Apr 1 17:14 ozone.master.nc
>>>>
>>>> inta:~/temp/3-atts/402a 1050> cat demo.wei-huang.0401a.ncl
>>>> begin
>>>> outfile = "out.nc"
>>>> ; system ("cp ozone.master.nc " + outfile) ; create fresh writable copy
>>>> system ("cat ozone.master.nc > " + outfile) ; create fresh writable copy
>>>> system ("chmod +w " + outfile) ; create fresh writable copy
>>>>
>>>> out = addfile (outfile, "w")
>>>>
>>>> out@last_update = systemfunc ("date")
>>>> out@last_update_program = "demo.global-atts.ncl version 2013-apr-01"
>>>> out@last_update_source_file = "ei.ozone.1979010106.nc"
>>>> print ("Update complete.")
>>>> end
>>>>
>>>> inta:~/temp/3-atts/402a 1051> $NCARG_ROOT/bin/ncl demo.wei-huang.0401a.ncl
>>>> Copyright (C) 1995-2013 - All Rights Reserved
>>>> University Corporation for Atmospheric Research
>>>> NCAR Command Language Version 6.1.2
>>>> The use of this software is governed by a License Agreement.
>>>> See http://www.ncl.ucar.edu/ for more details.
>>>> (0) Update complete.
>>>> Segmentation fault
>>>>
>>>> inta:~/temp/3-atts/402a 1052> ldd $NCARG_ROOT/bin/ncl
>>>> linux-vdso.so.1 => (0x00007fff7dbfd000)
>>>> libcurl.so.3 => /usr/lib64/libcurl.so.3 (0x0000003288400000)
>>>> libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003278600000)
>>>> libbz2.so.1 => /usr/lib64/libbz2.so.1 (0x000000328c000000)
>>>> libXrender.so.1 => /usr/lib64/libXrender.so.1 (0x000000327aa00000)
>>>> libX11.so.6 => /usr/lib64/libX11.so.6 (0x0000003278e00000)
>>>> libXext.so.6 => /usr/lib64/libXext.so.6 (0x0000003279a00000)
>>>> libm.so.6 => /lib64/libm.so.6 (0x0000003277a00000)
>>>> libdl.so.2 => /lib64/libdl.so.2 (0x0000003277e00000)
>>>> libstdc++.so.6 => /usr/local/gcc-4.7/lib/libstdc++.so.6 (0x00002b9f6bc6c000)
>>>> libgcc_s.so.1 => /usr/local/gcc-4.7/lib/libgcc_s.so.1 (0x00002b9f6bf76000)
>>>> libc.so.6 => /lib64/libc.so.6 (0x0000003277600000)
>>>> /lib64/ld-linux-x86-64.so.2 (0x0000003277200000)
>>>> libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x0000003282c00000)
>>>> libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x0000003282400000)
>>>> libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x0000003282000000)
>>>> libcom_err.so.2 => /lib64/libcom_err.so.2 (0x0000003281c00000)
>>>> libidn.so.11 => /usr/lib64/libidn.so.11 (0x0000003284000000)
>>>> libssl.so.6 => /lib64/libssl.so.6 (0x0000003288c00000)
>>>> libcrypto.so.6 => /lib64/libcrypto.so.6 (0x0000003288800000)
>>>> libz.so.1 => /lib64/libz.so.1 (0x0000003278200000)
>>>> libXau.so.6 => /usr/lib64/libXau.so.6 (0x0000003279600000)
>>>> libXdmcp.so.6 => /usr/lib64/libXdmcp.so.6 (0x0000003278a00000)
>>>> libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x0000003282800000)
>>>> libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x0000003281400000)
>>>> libresolv.so.2 => /lib64/libresolv.so.2 (0x0000003280c00000)
>>>> libselinux.so.1 => /lib64/libselinux.so.1 (0x000000327d200000)
>>>> libsepol.so.1 => /lib64/libsepol.so.1 (0x000000327ce00000)
>>>>
>>>> ========================================================
>>>>
>>>> mac56:~/ncl/bugs/3-atts/402a 35> uname -a
>>>> Darwin mac56 12.3.0 Darwin Kernel Version 12.3.0: Sun Jan 6 22:37:10
>>>> PST 2013; root:xnu-2050.22.13~1/RELEASE_X86_64 x86_64
>>>>
>>>> mac56:~/ncl/bugs/3-atts/402a 36> echo $NCARG_ROOT
>>>> /usr/local/ncl6.1.2
>>>>
>>>> mac56:~/ncl/bugs/3-atts/402a 37> ls -dgo $NCARG_ROOT/bin/ncl
>>>> -rwxr-xr-x 1 33112624 Feb 6 21:38 /usr/local/ncl6.1.2/bin/ncl
>>>>
>>>> mac56:~/ncl/bugs/3-atts/402a 38> ldd $NCARG_ROOT/bin/ncl
>>>> /usr/local/ncl6.1.2/bin/ncl:
>>>> /usr/lib/libcurl.4.dylib (compatibility version 7.0.0, current version 7.0.0)
>>>> /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
>>>> /usr/X11/lib/libXrender.1.dylib (compatibility version 5.0.0, current
>>>> version 5.0.0)
>>>> /usr/lib/libbz2.1.0.dylib (compatibility version 1.0.0, current version 1.0.5)
>>>> /usr/X11/lib/libXpm.4.dylib (compatibility version 16.0.0, current
>>>> version 16.0.0)
>>>> /usr/X11/lib/libX11.6.dylib (compatibility version 10.0.0, current
>>>> version 10.0.0)
>>>> /usr/X11/lib/libXext.6.dylib (compatibility version 11.0.0, current
>>>> version 11.0.0)
>>>> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
>>>> version 159.1.0)
>>>> /usr/local/lib/libstdc++.6.dylib (compatibility version 7.0.0,
>>>> current version 7.14.0)
>>>> /usr/local/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current
>>>> version 1.0.0)
>>>>
>>>> On Mon, Apr 1, 2013 at 9:35 PM, Wei Huang <huangwei@ucar.edu> wrote:
>>>>> Dave,
>>>>>
>>>>> I can not reproduce your problem.
>>>>>
>>>>> I only added:
>>>>> system ("chmod +w " + outfile)
>>>>> after system call in your script (as the original file is write protected).
>>>>>
>>>>> and it runs fine.
>>>>>
>>>>> I tested on Mac, with osx 10.8.3.
>>>>>
>>>>> Here is the full script I tested:
>>>>>
>>>>> ; 2013-apr-01 Test program reduced from era-update.ncl.
>>>>> ; Demonstrates seg fault when trying to overwrite global
>>>>> ; attributes in a Netcdf-4 classic file.
>>>>> ; By Dave Allured, NOAA/PSD/CIRES Climate Analysis Branch.
>>>>>
>>>>> begin
>>>>> outfile = "out.nc"
>>>>> ; system ("cp ozone.master.nc " + outfile) ; create fresh writable copy
>>>>> system ("cat ozone.master.nc > " + outfile) ; create fresh writable copy
>>>>> system ("chmod +w " + outfile) ; create fresh writable copy
>>>>>
>>>>> out = addfile (outfile, "w")
>>>>>
>>>>> out@last_update = systemfunc ("date")
>>>>> out@last_update_program = "demo.global-atts.ncl version 2013-apr-01"
>>>>> out@last_update_source_file = "ei.ozone.1979010106.nc"
>>>>> print ("Update complete.")
>>>>> end
>>>>>
>>>>>
>>>>> and part of ncdump -h out.nc:
>>>>>
>>>>> // global attributes:
>>>>> :title = "ERA-Interim Ozone mass mixing ratio, 4 times daily" ;
>>>>> :history = "Mon Apr 1 17:14:49 MDT 2013:\n",
>>>>> "Initial file made by era-create.ncl version 2013-apr-01 TEST" ;
>>>>> :dataset = "ERA-Interim" ;
>>>>> :version = "2.0" ;
>>>>> :references = "http://www.ecmwf.int/research/era/do/get/era-interim;\n",
>>>>> "D. P. Dee et al (2011), The ERA-Interim reanalysis:\n",
>>>>> " configuration and performance of the data assimilation system,\n",
>>>>> " Q.J.R. Meteorol. Soc., Vol. 137: 553-597, DOI: 10.1002/qj.828" ;
>>>>> :center = "European Center for Medium-Range Weather Forecasts (RSMC)" ;
>>>>> :metadata_source_file = "../ei.oper.an.pl.regn128sc.1979010100" ;
>>>>> :Conventions = "CF-1.6" ;
>>>>> :last_update = "Mon Apr 1 21:28:20 MDT 2013" ;
>>>>> :last_update_program = "demo.global-atts.ncl version 2013-apr-01" ;
>>>>> :last_update_source_file = "ei.ozone.1979010106.nc" ;
>>>>> }
>>>>>
>>>>>
>>>>> Wei
>>>>>
>>>>> huangwei@ucar.edu
>>>>> VETS/CISL
>>>>> National Center for Atmospheric Research
>>>>> P.O. Box 3000 (1850 Table Mesa Dr.)
>>>>> Boulder, CO 80307-3000 USA
>>>>> (303) 497-8924
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Apr 1, 2013, at 8:41 PM, Dave Allured - NOAA Affiliate <dave.allured@noaa.gov> wrote:
>>>>>
>>>>>> NCL support team,
>>>>>>
>>>>>> This program results in a seg fault error on program exit, after
>>>>>> attempting to overwrite some global attributes in a Netcdf-4 classic
>>>>>> file with HDF5 compression. Note that all three global attributes
>>>>>> exist previously in the target file, before running the test program.
>>>>>>
>>>>>> After the error, the target file is corrupted. This error is
>>>>>> strangely sensitive to the number and string lengths of the attributes
>>>>>> to be written. Can you please tell me what the problem is? Thank
>>>>>> you.
>>>>>>
>>>>>> begin
>>>>>> outfile = "out.nc"
>>>>>> system ("cat ozone.master.nc > " + outfile) ; create fresh writable copy
>>>>>>
>>>>>> out = addfile (outfile, "w")
>>>>>> out@last_update = systemfunc ("date")
>>>>>> out@last_update_program = "demo.global-atts.ncl version 2013-apr-01"
>>>>>> out@last_update_source_file = "ei.ozone.1979010106.nc"
>>>>>> print ("Update complete.")
>>>>>> end
>>>>>>
>>>>>> I get the same results with NCL versions 6.0.0 and 6.1.2. My system
>>>>>> is Mac OS 10.8.3 on a 64-bit Intel Mac desktop. Here is the console
>>>>>> output:
>>>>>>
>>>>>> mac56:~/ncl/bugs/3-atts/401i 491> ncl demo.global-atts.ncl
>>>>>> Copyright (C) 1995-2013 - All Rights Reserved
>>>>>> University Corporation for Atmospheric Research
>>>>>> NCAR Command Language Version 6.1.2
>>>>>> The use of this software is governed by a License Agreement.
>>>>>> See http://www.ncl.ucar.edu/ for more details.
>>>>>> (0) Update complete.
>>>>>> Segmentation fault
>>>>>>
>>>>>> mac56:~/ncl/bugs/3-atts/401i 492> ls -trgo
>>>>>> total 44136
>>>>>> -r--r--r-- 1 11294147 Apr 1 17:14 ozone.master.nc
>>>>>> -rw-r--r-- 1 570 Apr 1 19:51 demo.global-atts.ncl
>>>>>> -rw-r--r-- 1 11294201 Apr 1 20:16 out.nc
>>>>>>
>>>>>> mac56:~/ncl/bugs/3-atts/401i 493> ncdump -k ozone.master.nc
>>>>>> netCDF-4 classic model
>>>>>>
>>>>>> mac56:~/ncl/bugs/3-atts/401i 494> ncdump -k out.nc
>>>>>> ncdump: out.nc: NetCDF: Can't open HDF5 attribute
>>>>>>
>>>>>> mac56:~/ncl/bugs/3-atts/401i 495> uname -a
>>>>>> Darwin mac56 12.3.0 Darwin Kernel Version 12.3.0: Sun Jan 6 22:37:10
>>>>>> PST 2013; root:xnu-2050.22.13~1/RELEASE_X86_64 x86_64
>>>>>>
>>>>>> I uploaded the program and original Netcdf file (11 Mbytes) to
>>>>>> ftp.cgd.ucar.edu/incoming, as
>>>>>> "demo.global-atts.dave_allured.0401.zip". The original Netcdf file
>>>>>> was also made by NCL 6.1.2 and a different script. I can also send a
>>>>>> copy of the generating script and its input file, if you need it.
>>>>>> Please let me know. Thanks for taking a look at this.
>>>>>>
>>>>>> --Dave A.
>>>>>> NOAA/PSD/CIRES Climate Analysis Branch
>>>
>>> _______________________________________________
>>> ncl-talk mailing list
>>> List instructions, subscriber options, unsubscribe:
>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>
>>
>> _______________________________________________
>> ncl-talk mailing list
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Tue Apr 2 16:51:53 2013
This archive was generated by hypermail 2.1.8 : Tue Apr 02 2013 - 21:23:48 MDT