Re: version of netcdf4 libs in ncl 5.1.1

Jamie and I have been iterating back and forth on this offline, and have an answer from Unidata.

The issue was that Jamie wrote a NetCDF 4 classic file on a little endian machine using NetCDF 4.1.1, and was having trouble reading it using NCL V5.1.1. It reads fine using NCL V5.2.0.

This is because NCL V5.2.0 was built against NetCDF 4.1.1 and HDF5 1.8.4-patch1, which contains a bug fix for this problem. NCL V5.1.1 was built against NetCDF 4.0.1 and HDF5 1.8.4, as that's what was available at the time. This version hence has the bug.

Apparently the HDF5 bug in versions 1.8.4 and earlier affects little-endian machines as well. (Unidata initially thought it was just big-endian machines.)

The email snippet below is from Unidata, with the HDF5 developer response following:

------------------------------------------------------------------------------------------

> We have reports of a possibly different bug with the symptoms that
>
> - a file with more than 8 attributes of an object is written on a
> little-endian platform using version 1.8.4-patch1.
>
> - an attempt to read that file on a big-endian platform using version
> before 1.8.4-patch1 results in a seg fault.
>
> Is this just another symptom of the same bug (the contrapositive of the
> other bug :-), or is it a different bug?

Yes, that is the same bug. Basically, there are two populations as
far as the endianness of these values in the file goes:

1. All little-endian machines, and big-endian machines using 1.8.4-p1
  or later
2. Big-endian machines using 1.8.4 or earlier

Files written by one group cannot be read by the other. Files written
by group 2 do not conform to the file format spec and are technically
corrupt (though they can still be read just fine by group 2).

------------------------------------------------------------------------------------------

My suggestion is to not use NCL V5.1.1 (or any other software that was linked against HDF5 1.8.4 or earlier), if you have created a file using NetCDF 4.1.1 or later. If you upgrade to NCL V5.2.0, you should be able to read this file.

--Mary

On May 14, 2010, at 9:25 AM, Mary Haley wrote:
On Apr 29, 2010, at 9:18 AM, Mary Haley wrote:

> Hi Jamie,
>
> You're correct, NCL 5.2.0 was built against NetCDF 4.1.1. It's interesting that there's a backwards compatibility issue here.
>
> Once I have your file, I'll take a look, and see if Unidata has an explanation for the error.
>
> It's also possible that the incompatibility is in the HDF5 software, since NCL 5.1.1 was linked against HDF5 1.8.2 (I think it was), and NCL 5.2.0 was linked against HDF 1.8.4-patch1.
>
> --Mary
>
> On Apr 29, 2010, at 9:07 AM, Jamie Scott wrote:
>
>> Hi Mary,
>>
>> The file I'm trying to read was created with netcdf 4.1.1. It appears the problem is that ncl 5.1.1 is built with an earlier version.
>> (I also tried IDL, which used an earlier version of netcdf and it also failed) I downloaded the new ncl 5.2 and I'm able to read the file both on intel mac and x86_64! So I'm assuming that uses netcdf 4.1.1? I'm staging the file to our FTP server if you want to have a look. It's about 500MB, though. I'll let you know when it's ready...
>>
>> Here's the error message:
>>
>> [mac23:jscott/ncl/CCSM] jscott% ncl
>> Copyright (C) 1995-2009 - All Rights Reserved
>> University Corporation for Atmospheric Research
>> NCAR Command Language Version 5.1.1
>> The use of this software is governed by a License Agreement.
>> See http://www.ncl.ucar.edu/ for more details.
>> ncl 0> setfileoption("nc","Format","NetCDF4Classic")
>> ncl 1> f=addfile("/home/dmwork/Hoop/MERRA/slp.nc4Classic.1979.nc","r")
>> fatal:Can't open HDF5 attribute
>> fatal:Could not open (/home/dmwork/Hoop/MERRA/slp.nc4Classic.1979.nc)
>>
>> On Apr 29, 2010, at 7:39 AM, Mary Haley wrote:
>>
>>> Hi Jamie,
>>>
>>> NetCDF 5.1.1 was built against NetCDF 4.0.1, and classic NetCDF 4 should be available in that version.
>>>
>>> If you built NCL yourself from source code, however, the default is to build without NetCDF classic support,
>>> so make sure this was turned on if you built from source.
>>>
>>> Are you getting an error message? If so, what is it?
>>>
>>> Do you have a sample file that you can provide us with?
>>>
>>> --Mary
>>>
>>> On Apr 28, 2010, at 2:30 PM, Jamie Scott wrote:
>>>
>>>> Can anyone tell me what version of the netcdf4 library was used to
>>>> build ncl 5.1.1 (on intel Mac)?
>>>> I'm having trouble reading a netcdf4-classic file created with netcdf4
>>>> library version 4.1.1 and am wondering
>>>> if that is more recent than netcdf4 library in ncl 5.1.1.
>>>>
>>>> -Jamie
>>>>
>>>> _______________________________________________
>>>> ncl-talk mailing list
>>>> List instructions, subscriber options, unsubscribe:
>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>
>>
>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Fri May 14 14:41:12 2010