Re: Reading an odd-format ascii file

Gerry,

Your excerpt from the manual is a bit unclear. The data values are
coded in 5-character groups, hexadecimal, all on one long line of
text. They decode into signed integers.

To process in NCL, read in a file as all strings, identify the data
lines, break into 5-character substrings, then translate the hex
substrings to normal integers. You will have to do your own scaling
later, because I do not see the relevant information on scaling.

Special handling is needed for negative numbers with 5-character
hexadecimal format. The stringtoint function treats all such hex
strings as positive numbers. See the correction in the demo code
below. As usual, "there is more than one way to do this".

begin
infile = "ceilo.test"
data_line = 6 ; 1-based input line number
hex_len = 5 ; number of chars in each hex substring

lines = asciiread (infile, -1, "string")
hex_line = lines(data_line-1)

nchars = strlen (hex_line)
nvals = nchars / hex_len ; integer division truncates
print ("Number of 5-character hex values = " + nvals)

hex_strs = str_split_by_length (hex_line, hex_len)
ints = stringtoint ("0x" + hex_strs(0:nvals-1))
                 ; add prefix to get hexadecimal translation

max_neg = stringtoint ("0x80000")
ints = where (ints .lt. max_neg, ints, ints - (max_neg * 2))
                 ; correction for negative numbers

print (ints(0:3))
print (ints(766:))
print ("Min, max = " + min (ints) + ", " + max (ints))
end

--Dave

On Tue, Jan 22, 2013 at 12:55 PM, Gerry Creager - NOAA Affiliate
<gerry.creager@noaa.gov> wrote:
> I'd posted this some time in the past, but don't recall seeing a solution. I
> had to drop this but am now revisiting it for some upcoming field tests.
>
> I've a file from a Vaisala CL-31 ceilometer, in the "normal" ASCII
> representation that I need to read in. I intend to output the thing in a
> NetCDF format consistent with what the ARM project has for ceilometer data;
> I can handle that once I've got the data read in, I believe... I also want
> to be able to plot the backscatter data over time.
>
> I'm particularly stumped by the 4th line of data. The data format is
> described in the manual:
>
> 4TH LINE (5TH LINE OF MESSAGE NO. 2)
>
> Example: 00000111112222233333........(5 x 770 bytes)↵
>
> The two-way attenuated backscatter profile with sensitivity
>
> normalized units (100000·srad·km)-1 unless otherwise scaled by the
>
> SCALE parameter. Each sample is coded with a 20-bit HEX ASCII
>
> character set; msb nibble and bit first, 2's complement. The length of
>
> this line is equal to 5 times the length of the profile + 2.
>
>
> I'm thinking I should be able to read it in fairly easily to an array, and
> operate on the array, then write it back out as NetCDF with the values
> already corrected for extinguishment and scaling. However, I've been
> unsuccessful to date.
>
> The attached file is a single time sample.
>
> Thanks!
>
> gerry
>
> --
>
> Gerry Creager
> NSSL/CIMMS
> 405.325.6731
_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Tue Jan 22 14:05:56 2013