Basit,
Here is another method similar to Mary's. This was written before I
saw Mary's post. This may be of some value because of different
criteria for determining data vs. non-data.
The general approach for complex text like this is to first read the
whole file as a single large array of strings. Then you can more
easily work on the details in memory. Start with the following, and
also use str_strip to remove leading spaces because that will help
later.
infile = "sounding.2012.txt"
lines = str_strip (asciiread (infile, -1, "string"))
Next, find some way with NCL string functions to differentiate the
real data lines from non-data. There is a particular pattern here
that works nicely. If you look at the first non-blank character, data
lines start with a digit, and most other lines do not. The single
exception is handled separately.
char1 = str_get_cols (lines+"x", 0, 0)
nlines = dimsizes (lines)
non_flags = new (nlines, logical)
date_flags = new (nlines, logical)
do i = 0, nlines-1
non_flags(i) = ismissing (str_index_of_substr \
("0123456789", char1(i), 1))
date_flags(i) = .not. ismissing (str_index_of_substr \
(lines(i), "Observations", 1))
end do
num_flags = .not. (non_flags .or. date_flags)
print (num_flags + " " + str_get_cols (lines, 0, 40)) ; for testing only
(0) False 41024 OEJN Jeddah (King Abdul Aziz) Observations at
(1) False
(2) False ---------------------------------------------------
(3) False PRES HGHT TEMP DWPT RELH MIXR DRCT SK
(4) False hPa m C C % g/kg deg kno
(5) False ---------------------------------------------------
(6) True 1013.0 17 22.8 16.8 69 12.02 20
(7) True 1012.0 26 22.8 16.9 70 12.15 350
(8) True 1000.0 136 22.4 18.8 80 13.85 345
(9) True 984.0 275 21.3 18.4 83 13.73 345
(10) True 930.0 763 17.6 17.0 97 13.31 350
(11) False
(12) False Station information and sounding indices
(13) False
(14) False Station identifier: OEJN
(15) False Station number: 41024
(16) False Observation time: 120102/0000
(17) False Station latitude: 21.70
ETC.
So far so good. Now just get the data lines into a separate string
array. Then extract columns as shown in example asc4.txt and other
examples:
inds = ind (num_flags) ; list of subscripts for data lines only
data_lines = lines (inds) ; vector subscripting
print ("Number of data lines = " + dimsizes (data_lines))
pres = stringtofloat (str_get_field (data_lines, 1, " "))
hght = stringtoint (str_get_field (data_lines, 2, " "))
print (pres)
print (hght)
Variable: pres
Type: float
Total Size: 96 bytes
24 values
Number of Dimensions: 1
Dimensions and sizes: [24]
Coordinates:
(0) 1013
(1) 1012
(2) 1000
(3) 984
ETC.
Variable: hght
Type: integer
Total Size: 96 bytes
24 values
Number of Dimensions: 1
Dimensions and sizes: [24]
Coordinates:
(0) 17
(1) 26
(2) 136
(3) 275
ETC.
--Dave
On Thu, Jan 3, 2013 at 11:10 PM, BasitAli Khan
<BasitAli.Khan@kaust.edu.sa> wrote:
> Hi Dave,
>
> Its only one ascii file with one years data of 365 days with two soundings
> each day. Every sounding has both head and footer just like the snippet of
> the data I copied in my first email. The number of rows (pressure levels)
> in each sounding also varies on different days. Since the header and
> footer repeats (month, day, time also varies in some part of the header)
> for every sounding, none of the example including asc4.txt applies to my
> case.
>
> I will greatly appreciate if you could give some ideas, clues, advise to
> handle this dataset.
>
> Best regards,
>
> ----
> Basit A. Khan, Ph.D.
>
>
> On 1/4/13 7:01 AM, "Dave Allured - NOAA Affiliate" <dave.allured@noaa.gov>
> wrote:
>
>>Basit,
>>
>>It is important to know what exactly is in a single file. Is there
>>just one header and one footer per file? It is not clear whether you
>>first posted a single ascii file, or two or three files pasted
>>together.
>>
>>If there is only one header and one footer, are the number of header
>>lines and footer lines always the same? If the answer is "yes" to all
>>of these, then I think you can use a variation of example asc4.txt on
>>the Read ASCII example page. Will this work?
>>
>>http://www.ncl.ucar.edu/Applications/read_ascii.shtml#asc4
>>
>>--Dave
>>(Please reply to list only!)
>>
>>On Wed, Jan 2, 2013 at 5:49 AM, BasitAli Khan
>><BasitAli.Khan@kaust.edu.sa> wrote:
>>> Dear all,
>>>
>>> I have vertical sounding data of the following type (upper air - Wyoming
>>> Univ.) that i want to load and process in ncl. Its one year data and i
>>>have
>>> two soundings for each day. For every sounding there is header and
>>>footer on
>>> top and bottom of the numeric data. I am just wondering as how can i
>>>load
>>> ONLY numeric data fields into variables while ignoring all the header
>>>and
>>> footer info. I am using asciiread function to load this file as string
>>> datatype but don't know how to tell ncl to exclude any redundant info.
>>> Unfortunately, I could not find any relevant examples on
>>> http://www.ncl.ucar.edu/Applications/read_ascii.shtml and would greatly
>>> appreciate if anybody could advise a solution.
>>>
>>> Best regards,
>>> Basit
>>>
>>>
>>> 41024 OEJN Jeddah (King Abdul Aziz) Observations at 00Z 02 Jan 2012
>>>
>>>
>>>-------------------------------------------------------------------------
>>>----
>>> PRES HGHT TEMP DWPT RELH MIXR DRCT SKNT THTA THTE
>>> THTV
>>> hPa m C C % g/kg deg knot K K
>>> K
>>>
>>>-------------------------------------------------------------------------
>>>----
>>> 1013.0 17 22.8 16.8 69 12.02 20 10 294.9 329.5
>>> 297.0
>>> 1012.0 26 22.8 16.9 70 12.15 350 14 294.9 330.0
>>> 297.1
>>> 1000.0 136 22.4 18.8 80 13.85 345 16 295.6 335.5
>>> 298.0
>>> 984.0 275 21.3 18.4 83 13.73 345 20 295.8 335.5
>>> 298.3
>>> 930.0 763 17.6 17.0 97 13.31 350 14 296.8 335.4
>>> 299.2
>>>
>>> Station information and sounding indices
>>>
>>> Station identifier: OEJN
>>> Station number: 41024
>>> Observation time: 120102/0000
>>> Station latitude: 21.70
>>> Station longitude: 39.18
>>> Station elevation: 17.0
>>> 41024 OEJN Jeddah (King Abdul Aziz) Observations at 12Z 02 Jan 2012
>>>
>>>
>>>-------------------------------------------------------------------------
>>>----
>>> PRES HGHT TEMP DWPT RELH MIXR DRCT SKNT THTA THTE
>>> THTV
>>> hPa m C C % g/kg deg knot K K
>>> K
>>>
>>>-------------------------------------------------------------------------
>>>----
>>> 1012.0 17 27.0 14.0 45 10.02 320 14 299.1 328.7
>>> 300.9
>>> 1011.0 26 24.6 12.6 47 9.14 322 15 296.8 323.6
>>> 298.5
>>> 1004.0 90 24.2 13.5 51 9.77 340 19 297.0 325.6
>>> 298.8
>>> 1000.0 126 24.0 14.0 54 10.14 335 19 297.1 326.8
>>> 299.0
>>> 943.0 633 18.7 12.7 68 9.89 325 16 296.8 325.7
>>> 298.6
>>> 927.0 781 17.2 12.3 73 9.81 340 18 296.7 325.4
>>> 298.4
>>> 925.0 800 17.0 12.3 74 9.80 345 18 296.7 325.3
>>> 298.4
>>> 907.0 967 15.7 11.8 78 9.71 345 14 297.0 325.4
>>> 298.7
>>> 893.0 1100 14.6 11.5 82 9.63 20 10 297.2 325.4
>>> 298.9
>>> 887.0 1157 14.8 10.5 75 9.06 9 5 298.0 324.7
>>> 299.6
>>> Station information and sounding indices
>>>
>>> Station identifier: OEJN
>>> Station number: 41024
>>> Observation time: 120102/1200
>>> Station latitude: 21.70
>>> Station longitude: 39.18
>>> Station elevation: 17.0
>>> Showalter index: 10.08
>>> Lifted index: 5.13
>>> LIFT computed using virtual temperature: 4.91
>>>
>>> 41024 OEJN Jeddah (King Abdul Aziz) Observations at 00Z 03 Jan 2012
>>>
>>>
>>>-------------------------------------------------------------------------
>>>----
>>> PRES HGHT TEMP DWPT RELH MIXR DRCT SKNT THTA THTE
>>> THTV
>>> hPa m C C % g/kg deg knot K K
>>> K
>>>
>>>-------------------------------------------------------------------------
>>>----
>>> 1013.0 17 20.0 13.0 64 9.37 50 5 292.1 318.9
>>> 293.7
>>> 1011.0 35 21.6 15.6 69 11.14 50 5 293.8 325.9
>>> 295.8
>>> 1000.0 134 21.4 14.4 64 10.41 50 4 294.6 324.6
>>> 296.4
>>> 964.0 450 18.9 14.5 76 10.91 40 2 295.1 326.6
>>> 297.0
>>> 925.0 806 16.0 14.7 92 11.49 5 4 295.7 328.9
>>> 297.7
>>> 915.0 898 15.0 14.3 96 11.32 335 5 295.6 328.3
>>> 297.6
>>> 893.0 1105 14.6 13.1 91 10.72 268 8 297.2 328.5
>>> 299.1
>>> 876.0 1268 15.7 13.1 85 10.93 215 10 300.0 332.2
>>> 301.9
>>> 868.0 1346 16.2 13.1 82 11.03 197 15 301.3 334.0
>>> 303.3
>>>
>>>
>>> ________________________________
_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Fri Jan 4 14:46:01 2013
This archive was generated by hypermail 2.1.8 : Fri Jan 04 2013 - 15:34:10 MST