Hi Mary and Dave,
Thanks very much indeed for your suggestions and script. Based on this, i
think i can develop some logic and put-together a script that would read
only the data and date info from the header.
Cheers,
----
Basit A. Khan, Ph.D.
Postdoctoral Research Fellow
Division of Physical Sciences & Engineering
Office# 3204, Level 3, Building 1,
King Abdullah University of Science & Technology
4700 King Abdullah Blvd, Box 2753, Thuwal 23955 6900,
Kingdom of Saudi Arabia.
Office: +966(0)2 808 0276, Mobile: +966(0)5 0860 3617
E-mail: basitali.khan@kaust.edu.sa
Skype name: basit.a.khan
On 1/5/13 12:45 AM, "Dave Allured - NOAA Affiliate"
<dave.allured@noaa.gov> wrote:
>Basit,
>
>Here is another method similar to Mary's. This was written before I
>saw Mary's post. This may be of some value because of different
>criteria for determining data vs. non-data.
>
>The general approach for complex text like this is to first read the
>whole file as a single large array of strings. Then you can more
>easily work on the details in memory. Start with the following, and
>also use str_strip to remove leading spaces because that will help
>later.
>
> infile = "sounding.2012.txt"
> lines = str_strip (asciiread (infile, -1, "string"))
>
>Next, find some way with NCL string functions to differentiate the
>real data lines from non-data. There is a particular pattern here
>that works nicely. If you look at the first non-blank character, data
>lines start with a digit, and most other lines do not. The single
>exception is handled separately.
>
> char1 = str_get_cols (lines+"x", 0, 0)
> nlines = dimsizes (lines)
> non_flags = new (nlines, logical)
> date_flags = new (nlines, logical)
>
> do i = 0, nlines-1
> non_flags(i) = ismissing (str_index_of_substr \
> ("0123456789", char1(i), 1))
> date_flags(i) = .not. ismissing (str_index_of_substr \
> (lines(i), "Observations", 1))
> end do
>
> num_flags = .not. (non_flags .or. date_flags)
> print (num_flags + " " + str_get_cols (lines, 0, 40)) ; for testing
>only
>
>(0) False 41024 OEJN Jeddah (King Abdul Aziz) Observations at
>(1) False
>(2) False ---------------------------------------------------
>(3) False PRES HGHT TEMP DWPT RELH MIXR DRCT SK
>(4) False hPa m C C % g/kg deg kno
>(5) False ---------------------------------------------------
>(6) True 1013.0 17 22.8 16.8 69 12.02 20
>(7) True 1012.0 26 22.8 16.9 70 12.15 350
>(8) True 1000.0 136 22.4 18.8 80 13.85 345
>(9) True 984.0 275 21.3 18.4 83 13.73 345
>(10) True 930.0 763 17.6 17.0 97 13.31 350
>(11) False
>(12) False Station information and sounding indices
>(13) False
>(14) False Station identifier: OEJN
>(15) False Station number: 41024
>(16) False Observation time: 120102/0000
>(17) False Station latitude: 21.70
>ETC.
>
>So far so good. Now just get the data lines into a separate string
>array. Then extract columns as shown in example asc4.txt and other
>examples:
>
> inds = ind (num_flags) ; list of subscripts for data lines only
> data_lines = lines (inds) ; vector subscripting
> print ("Number of data lines = " + dimsizes (data_lines))
>
> pres = stringtofloat (str_get_field (data_lines, 1, " "))
> hght = stringtoint (str_get_field (data_lines, 2, " "))
>
> print (pres)
> print (hght)
>
>Variable: pres
>Type: float
>Total Size: 96 bytes
> 24 values
>Number of Dimensions: 1
>Dimensions and sizes: [24]
>Coordinates:
>(0) 1013
>(1) 1012
>(2) 1000
>(3) 984
>ETC.
>
>Variable: hght
>Type: integer
>Total Size: 96 bytes
> 24 values
>Number of Dimensions: 1
>Dimensions and sizes: [24]
>Coordinates:
>(0) 17
>(1) 26
>(2) 136
>(3) 275
>ETC.
>
>--Dave
>
>On Thu, Jan 3, 2013 at 11:10 PM, BasitAli Khan
><BasitAli.Khan@kaust.edu.sa> wrote:
>> Hi Dave,
>>
>> Its only one ascii file with one years data of 365 days with two
>>soundings
>> each day. Every sounding has both head and footer just like the snippet
>>of
>> the data I copied in my first email. The number of rows (pressure
>>levels)
>> in each sounding also varies on different days. Since the header and
>> footer repeats (month, day, time also varies in some part of the header)
>> for every sounding, none of the example including asc4.txt applies to my
>> case.
>>
>> I will greatly appreciate if you could give some ideas, clues, advise to
>> handle this dataset.
>>
>> Best regards,
>>
>> ----
>> Basit A. Khan, Ph.D.
>>
>>
>> On 1/4/13 7:01 AM, "Dave Allured - NOAA Affiliate"
>><dave.allured@noaa.gov>
>> wrote:
>>
>>>Basit,
>>>
>>>It is important to know what exactly is in a single file. Is there
>>>just one header and one footer per file? It is not clear whether you
>>>first posted a single ascii file, or two or three files pasted
>>>together.
>>>
>>>If there is only one header and one footer, are the number of header
>>>lines and footer lines always the same? If the answer is "yes" to all
>>>of these, then I think you can use a variation of example asc4.txt on
>>>the Read ASCII example page. Will this work?
>>>
>>>http://www.ncl.ucar.edu/Applications/read_ascii.shtml#asc4
>>>
>>>--Dave
>>>(Please reply to list only!)
>>>
>>>On Wed, Jan 2, 2013 at 5:49 AM, BasitAli Khan
>>><BasitAli.Khan@kaust.edu.sa> wrote:
>>>> Dear all,
>>>>
>>>> I have vertical sounding data of the following type (upper air -
>>>>Wyoming
>>>> Univ.) that i want to load and process in ncl. Its one year data and i
>>>>have
>>>> two soundings for each day. For every sounding there is header and
>>>>footer on
>>>> top and bottom of the numeric data. I am just wondering as how can i
>>>>load
>>>> ONLY numeric data fields into variables while ignoring all the header
>>>>and
>>>> footer info. I am using asciiread function to load this file as
>>>>string
>>>> datatype but don't know how to tell ncl to exclude any redundant info.
>>>> Unfortunately, I could not find any relevant examples on
>>>> http://www.ncl.ucar.edu/Applications/read_ascii.shtml and would
>>>>greatly
>>>> appreciate if anybody could advise a solution.
>>>>
>>>> Best regards,
>>>> Basit
>>>>
>>>>
>>>> 41024 OEJN Jeddah (King Abdul Aziz) Observations at 00Z 02 Jan 2012
>>>>
>>>>
>>>>-----------------------------------------------------------------------
>>>>--
>>>>----
>>>> PRES HGHT TEMP DWPT RELH MIXR DRCT SKNT THTA THTE
>>>> THTV
>>>> hPa m C C % g/kg deg knot K K
>>>> K
>>>>
>>>>-----------------------------------------------------------------------
>>>>--
>>>>----
>>>> 1013.0 17 22.8 16.8 69 12.02 20 10 294.9 329.5
>>>> 297.0
>>>> 1012.0 26 22.8 16.9 70 12.15 350 14 294.9 330.0
>>>> 297.1
>>>> 1000.0 136 22.4 18.8 80 13.85 345 16 295.6 335.5
>>>> 298.0
>>>> 984.0 275 21.3 18.4 83 13.73 345 20 295.8 335.5
>>>> 298.3
>>>> 930.0 763 17.6 17.0 97 13.31 350 14 296.8 335.4
>>>> 299.2
>>>>
>>>> Station information and sounding indices
>>>>
>>>> Station identifier: OEJN
>>>> Station number: 41024
>>>> Observation time: 120102/0000
>>>> Station latitude: 21.70
>>>> Station longitude: 39.18
>>>> Station elevation: 17.0
>>>> 41024 OEJN Jeddah (King Abdul Aziz) Observations at 12Z 02 Jan 2012
>>>>
>>>>
>>>>-----------------------------------------------------------------------
>>>>--
>>>>----
>>>> PRES HGHT TEMP DWPT RELH MIXR DRCT SKNT THTA THTE
>>>> THTV
>>>> hPa m C C % g/kg deg knot K K
>>>> K
>>>>
>>>>-----------------------------------------------------------------------
>>>>--
>>>>----
>>>> 1012.0 17 27.0 14.0 45 10.02 320 14 299.1 328.7
>>>> 300.9
>>>> 1011.0 26 24.6 12.6 47 9.14 322 15 296.8 323.6
>>>> 298.5
>>>> 1004.0 90 24.2 13.5 51 9.77 340 19 297.0 325.6
>>>> 298.8
>>>> 1000.0 126 24.0 14.0 54 10.14 335 19 297.1 326.8
>>>> 299.0
>>>> 943.0 633 18.7 12.7 68 9.89 325 16 296.8 325.7
>>>> 298.6
>>>> 927.0 781 17.2 12.3 73 9.81 340 18 296.7 325.4
>>>> 298.4
>>>> 925.0 800 17.0 12.3 74 9.80 345 18 296.7 325.3
>>>> 298.4
>>>> 907.0 967 15.7 11.8 78 9.71 345 14 297.0 325.4
>>>> 298.7
>>>> 893.0 1100 14.6 11.5 82 9.63 20 10 297.2 325.4
>>>> 298.9
>>>> 887.0 1157 14.8 10.5 75 9.06 9 5 298.0 324.7
>>>> 299.6
>>>> Station information and sounding indices
>>>>
>>>> Station identifier: OEJN
>>>> Station number: 41024
>>>> Observation time: 120102/1200
>>>> Station latitude: 21.70
>>>> Station longitude: 39.18
>>>> Station elevation: 17.0
>>>> Showalter index: 10.08
>>>> Lifted index: 5.13
>>>> LIFT computed using virtual temperature: 4.91
>>>>
>>>> 41024 OEJN Jeddah (King Abdul Aziz) Observations at 00Z 03 Jan 2012
>>>>
>>>>
>>>>-----------------------------------------------------------------------
>>>>--
>>>>----
>>>> PRES HGHT TEMP DWPT RELH MIXR DRCT SKNT THTA THTE
>>>> THTV
>>>> hPa m C C % g/kg deg knot K K
>>>> K
>>>>
>>>>-----------------------------------------------------------------------
>>>>--
>>>>----
>>>> 1013.0 17 20.0 13.0 64 9.37 50 5 292.1 318.9
>>>> 293.7
>>>> 1011.0 35 21.6 15.6 69 11.14 50 5 293.8 325.9
>>>> 295.8
>>>> 1000.0 134 21.4 14.4 64 10.41 50 4 294.6 324.6
>>>> 296.4
>>>> 964.0 450 18.9 14.5 76 10.91 40 2 295.1 326.6
>>>> 297.0
>>>> 925.0 806 16.0 14.7 92 11.49 5 4 295.7 328.9
>>>> 297.7
>>>> 915.0 898 15.0 14.3 96 11.32 335 5 295.6 328.3
>>>> 297.6
>>>> 893.0 1105 14.6 13.1 91 10.72 268 8 297.2 328.5
>>>> 299.1
>>>> 876.0 1268 15.7 13.1 85 10.93 215 10 300.0 332.2
>>>> 301.9
>>>> 868.0 1346 16.2 13.1 82 11.03 197 15 301.3 334.0
>>>> 303.3
>>>>
>>>>
>>>> ________________________________
________________________________
This message and its contents including attachments are intended solely for the original recipient. If you are not the intended recipient or have received this message in error, please notify me immediately and delete this message from your computer system. Any unauthorized use or distribution is prohibited. Please consider the environment before printing this email.
_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Fri Jan 4 22:39:37 2013
This archive was generated by hypermail 2.1.8 : Mon Jan 07 2013 - 08:57:08 MST