Re: Byte array

From: Dave Allured - NOAA Affiliate <dave.allured_at_nyahnyahspammersnyahnyah>
Date: Mon Jul 29 2013 - 12:42:14 MDT

Marston and Dave B,

Marston originally said, "... in the CloudSat docs, the Data_quality
flag is set to 0 if the data is good." If the _FillValue attribute is
present and zero after NCL reads this hdf4-eos file, this is totally
inconsistent with this documentation, and there is an error somewhere.

Possibilities include a special case problem in the NCL driver, the
hdf4-eos file was modified or produced under different rules, damaged
file, mistake in the documentation, subtle problem in the user NCL
script, etc.

I think that Dave B's suggestion to delete the _FillValue attribute in
NCL memory will mitigate the problem, but I would still wonder how and
why _FillValue was set to zero. Marston, do you have any software,
other than NCL, to display the file's metadata and check for the
expected missing value a different way? I am not experienced with
HDF4, so I can only raise suspicions, FWIW.

--Dave A.

On Mon, Jul 29, 2013 at 12:00 PM, David Brown <dbrown@ucar.edu> wrote:
> Hi Marston,
>
> You cannot test a value that is set to the _FillValue for equality using e.g.: Data_quality .eq. 0. Instead you should use the 'ismissing' function, for example:
>
> where (ismissing(Data_quality), test_var = test_var@_FillValue, test_var)
>
> (assumes test_var is another variable in the file that has the same dimensionality as Data_quality)
>
> The other option in this case might be to delete the _FillValue attribute:
>
> delete (Data_quality@_FillValue)
> where (Data_quality .eq. 0, test_var = test_var@_FillValue, test_var)
>
> This is in no way specific to the byte type.
>
> It is a feature of ncdump that values set to the _FillValue are printed as '_'. Again this is not specific to any type. If you don't like this you could use ncl_filedump instead.
> -dave
>
>
>
> On Jul 29, 2013, at 11:09 AM, Marston Johnston <shejo284@gmail.com> wrote:
>
>>
>> Hi,
>>
>> I'm working with CloudSat hdf4-eos files using NCL version 6.1.
>> In the CloudSat file, the Data_Quality variable in the hdf file is of type "byte".
>> Is there a function to check which byte is set? For example, how do I check other variable arrays using mask or where, with the condition that Data_quality is 0? I tried the function byte2flt, but this fails. The problem is that in the CloudSat docs, the Data_quality flag is set to 0 if the data is good. Otherwise the data is bad. After NCL reads the variable, the _FillValue is set to 0.
>>
>> Variable: Data_quality
>> Type: byte
>> Total Size: 20820 bytes
>> 20820 values
>> Number of Dimensions: 1
>> Dimensions and sizes: [nray_2C_ICE | 20820]
>> Coordinates:
>> Number Of Attributes: 3
>> _FillValue : 0
>> unsigned : True
>> hdfeos_name : Data_quality
>>
>> Another strange thing is that when the hdd file is converted to netcdf, the ncdump shows much of, if not all of the data as containing: '-', not values.
>>
>> Appreciate little info on how to treat byte arrays.
>>
>> /M
_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Mon Jul 29 12:42:21 2013

This archive was generated by hypermail 2.1.8 : Thu Aug 01 2013 - 15:55:04 MDT