Re: reading ascii vertical sounding data (Upper air data from Univ. of Wyoming)

From: BasitAli Khan <BasitAli.Khan_at_nyahnyahspammersnyahnyah>
Date: Fri Jan 04 2013 - 22:39:24 MST

Hi Mary and Dave,

Thanks very much indeed for your suggestions and script. Based on this, i
think i can develop some logic and put-together a script that would read
only the data and date info from the header.

Cheers,

----
Basit A. Khan, Ph.D.
Postdoctoral Research Fellow
Division of Physical Sciences & Engineering
Office# 3204, Level 3, Building 1,
King Abdullah University of Science & Technology
4700 King Abdullah Blvd, Box 2753, Thuwal 23955 6900,
Kingdom of Saudi Arabia.
Office: +966(0)2 808 0276,  Mobile: +966(0)5 0860 3617
E-mail: basitali.khan@kaust.edu.sa
Skype name: basit.a.khan
On 1/5/13 12:45 AM, "Dave Allured - NOAA Affiliate"
<dave.allured@noaa.gov> wrote:
>Basit,
>
>Here is another method similar to Mary's.  This was written before I
>saw Mary's post.  This may be of some value because of different
>criteria for determining data vs. non-data.
>
>The general approach for complex text like this is to first read the
>whole file as a single large array of strings.  Then you can more
>easily work on the details in memory.  Start with the following, and
>also use str_strip to remove leading spaces because that will help
>later.
>
>  infile = "sounding.2012.txt"
>  lines = str_strip (asciiread (infile, -1, "string"))
>
>Next, find some way with NCL string functions to differentiate the
>real data lines from non-data.  There is a particular pattern here
>that works nicely.  If you look at the first non-blank character, data
>lines start with a digit, and most other lines do not.  The single
>exception is handled separately.
>
>  char1 = str_get_cols (lines+"x", 0, 0)
>  nlines = dimsizes (lines)
>  non_flags  = new (nlines, logical)
>  date_flags = new (nlines, logical)
>
>  do i = 0, nlines-1
>    non_flags(i) = ismissing (str_index_of_substr \
>      ("0123456789", char1(i), 1))
>    date_flags(i) = .not. ismissing (str_index_of_substr \
>      (lines(i), "Observations", 1))
>  end do
>
>  num_flags = .not. (non_flags .or. date_flags)
>  print (num_flags + " " + str_get_cols (lines, 0, 40))  ; for testing
>only
>
>(0)    False 41024 OEJN Jeddah (King Abdul Aziz) Observations at
>(1)    False
>(2)    False ---------------------------------------------------
>(3)    False PRES   HGHT   TEMP   DWPT   RELH   MIXR   DRCT   SK
>(4)    False hPa     m      C      C      %    g/kg    deg   kno
>(5)    False ---------------------------------------------------
>(6)    True 1013.0     17   22.8   16.8     69  12.02     20
>(7)    True 1012.0     26   22.8   16.9     70  12.15    350
>(8)    True 1000.0    136   22.4   18.8     80  13.85    345
>(9)    True 984.0    275   21.3   18.4     83  13.73    345
>(10)   True 930.0    763   17.6   17.0     97  13.31    350
>(11)   False
>(12)   False Station information and sounding indices
>(13)   False
>(14)   False Station identifier: OEJN
>(15)   False Station number: 41024
>(16)   False Observation time: 120102/0000
>(17)   False Station latitude: 21.70
>ETC.
>
>So far so good.  Now just get the data lines into a separate string
>array.  Then extract columns as shown in example asc4.txt and other
>examples:
>
>  inds = ind (num_flags)   ; list of subscripts for data lines only
>  data_lines = lines (inds)   ; vector subscripting
>  print ("Number of data lines = " + dimsizes (data_lines))
>
>  pres = stringtofloat (str_get_field (data_lines, 1, " "))
>  hght = stringtoint   (str_get_field (data_lines, 2, " "))
>
>  print (pres)
>  print (hght)
>
>Variable: pres
>Type: float
>Total Size: 96 bytes
>            24 values
>Number of Dimensions: 1
>Dimensions and sizes:  [24]
>Coordinates:
>(0)    1013
>(1)    1012
>(2)    1000
>(3)    984
>ETC.
>
>Variable: hght
>Type: integer
>Total Size: 96 bytes
>            24 values
>Number of Dimensions: 1
>Dimensions and sizes:  [24]
>Coordinates:
>(0)    17
>(1)    26
>(2)    136
>(3)    275
>ETC.
>
>--Dave
>
>On Thu, Jan 3, 2013 at 11:10 PM, BasitAli Khan
><BasitAli.Khan@kaust.edu.sa> wrote:
>> Hi Dave,
>>
>> Its only one ascii file with one years data of 365 days with two
>>soundings
>> each day. Every sounding has both head and footer just like the snippet
>>of
>> the data I copied in my first email. The number of rows (pressure
>>levels)
>> in each sounding also varies on different days. Since the header and
>> footer repeats (month, day, time also varies in some part of the header)
>> for every sounding, none of the example including asc4.txt applies to my
>> case.
>>
>> I will greatly appreciate if you could give some ideas, clues, advise to
>> handle this dataset.
>>
>> Best regards,
>>
>> ----
>> Basit A. Khan, Ph.D.
>>
>>
>> On 1/4/13 7:01 AM, "Dave Allured - NOAA Affiliate"
>><dave.allured@noaa.gov>
>> wrote:
>>
>>>Basit,
>>>
>>>It is important to know what exactly is in a single file.  Is there
>>>just one header and one footer per file?  It is not clear whether you
>>>first posted a single ascii file, or two or three files pasted
>>>together.
>>>
>>>If there is only one header and one footer, are the number of header
>>>lines and footer lines always the same?  If the answer is "yes" to all
>>>of these, then I think you can use a variation of example asc4.txt on
>>>the Read ASCII example page.  Will this work?
>>>
>>>http://www.ncl.ucar.edu/Applications/read_ascii.shtml#asc4
>>>
>>>--Dave
>>>(Please reply to list only!)
>>>
>>>On Wed, Jan 2, 2013 at 5:49 AM, BasitAli Khan
>>><BasitAli.Khan@kaust.edu.sa> wrote:
>>>> Dear all,
>>>>
>>>> I have vertical sounding data of the following type (upper air -
>>>>Wyoming
>>>> Univ.) that i want to load and process in ncl. Its one year data and i
>>>>have
>>>> two soundings for each day. For every sounding there is header and
>>>>footer on
>>>> top and bottom of the numeric data. I am just wondering as how can i
>>>>load
>>>> ONLY numeric data fields into variables while ignoring all the header
>>>>and
>>>> footer info.  I am using asciiread function to load this file as
>>>>string
>>>> datatype but don't know how to tell ncl to exclude any redundant info.
>>>> Unfortunately, I could not find any relevant examples on
>>>> http://www.ncl.ucar.edu/Applications/read_ascii.shtml and would
>>>>greatly
>>>> appreciate if anybody could advise a solution.
>>>>
>>>> Best regards,
>>>> Basit
>>>>
>>>>
>>>> 41024 OEJN Jeddah (King Abdul Aziz) Observations at 00Z 02 Jan 2012
>>>>
>>>>
>>>>-----------------------------------------------------------------------
>>>>--
>>>>----
>>>>    PRES   HGHT   TEMP   DWPT   RELH   MIXR   DRCT   SKNT   THTA   THTE
>>>> THTV
>>>>     hPa     m      C      C      %    g/kg    deg   knot     K      K
>>>>   K
>>>>
>>>>-----------------------------------------------------------------------
>>>>--
>>>>----
>>>>  1013.0     17   22.8   16.8     69  12.02     20     10  294.9  329.5
>>>> 297.0
>>>>  1012.0     26   22.8   16.9     70  12.15    350     14  294.9  330.0
>>>> 297.1
>>>>  1000.0    136   22.4   18.8     80  13.85    345     16  295.6  335.5
>>>> 298.0
>>>>   984.0    275   21.3   18.4     83  13.73    345     20  295.8  335.5
>>>> 298.3
>>>>   930.0    763   17.6   17.0     97  13.31    350     14  296.8  335.4
>>>> 299.2
>>>>
>>>> Station information and sounding indices
>>>>
>>>>                          Station identifier: OEJN
>>>>                              Station number: 41024
>>>>                            Observation time: 120102/0000
>>>>                            Station latitude: 21.70
>>>>                           Station longitude: 39.18
>>>>                           Station elevation: 17.0
>>>> 41024 OEJN Jeddah (King Abdul Aziz) Observations at 12Z 02 Jan 2012
>>>>
>>>>
>>>>-----------------------------------------------------------------------
>>>>--
>>>>----
>>>>    PRES   HGHT   TEMP   DWPT   RELH   MIXR   DRCT   SKNT   THTA   THTE
>>>> THTV
>>>>     hPa     m      C      C      %    g/kg    deg   knot     K      K
>>>>   K
>>>>
>>>>-----------------------------------------------------------------------
>>>>--
>>>>----
>>>>  1012.0     17   27.0   14.0     45  10.02    320     14  299.1  328.7
>>>> 300.9
>>>>  1011.0     26   24.6   12.6     47   9.14    322     15  296.8  323.6
>>>> 298.5
>>>>  1004.0     90   24.2   13.5     51   9.77    340     19  297.0  325.6
>>>> 298.8
>>>>  1000.0    126   24.0   14.0     54  10.14    335     19  297.1  326.8
>>>> 299.0
>>>>   943.0    633   18.7   12.7     68   9.89    325     16  296.8  325.7
>>>> 298.6
>>>>   927.0    781   17.2   12.3     73   9.81    340     18  296.7  325.4
>>>> 298.4
>>>>   925.0    800   17.0   12.3     74   9.80    345     18  296.7  325.3
>>>> 298.4
>>>>   907.0    967   15.7   11.8     78   9.71    345     14  297.0  325.4
>>>> 298.7
>>>>   893.0   1100   14.6   11.5     82   9.63     20     10  297.2  325.4
>>>> 298.9
>>>>   887.0   1157   14.8   10.5     75   9.06      9      5  298.0  324.7
>>>> 299.6
>>>> Station information and sounding indices
>>>>
>>>>                          Station identifier: OEJN
>>>>                              Station number: 41024
>>>>                            Observation time: 120102/1200
>>>>                            Station latitude: 21.70
>>>>                           Station longitude: 39.18
>>>>                           Station elevation: 17.0
>>>>                             Showalter index: 10.08
>>>>                                Lifted index: 5.13
>>>>     LIFT computed using virtual temperature: 4.91
>>>>
>>>> 41024 OEJN Jeddah (King Abdul Aziz) Observations at 00Z 03 Jan 2012
>>>>
>>>>
>>>>-----------------------------------------------------------------------
>>>>--
>>>>----
>>>>    PRES   HGHT   TEMP   DWPT   RELH   MIXR   DRCT   SKNT   THTA   THTE
>>>> THTV
>>>>     hPa     m      C      C      %    g/kg    deg   knot     K      K
>>>>   K
>>>>
>>>>-----------------------------------------------------------------------
>>>>--
>>>>----
>>>>  1013.0     17   20.0   13.0     64   9.37     50      5  292.1  318.9
>>>> 293.7
>>>>  1011.0     35   21.6   15.6     69  11.14     50      5  293.8  325.9
>>>> 295.8
>>>>  1000.0    134   21.4   14.4     64  10.41     50      4  294.6  324.6
>>>> 296.4
>>>>   964.0    450   18.9   14.5     76  10.91     40      2  295.1  326.6
>>>> 297.0
>>>>   925.0    806   16.0   14.7     92  11.49      5      4  295.7  328.9
>>>> 297.7
>>>>   915.0    898   15.0   14.3     96  11.32    335      5  295.6  328.3
>>>> 297.6
>>>>   893.0   1105   14.6   13.1     91  10.72    268      8  297.2  328.5
>>>> 299.1
>>>>   876.0   1268   15.7   13.1     85  10.93    215     10  300.0  332.2
>>>> 301.9
>>>>   868.0   1346   16.2   13.1     82  11.03    197     15  301.3  334.0
>>>> 303.3
>>>>
>>>>
>>>> ________________________________
________________________________
This message and its contents including attachments are intended solely for the original recipient. If you are not the intended recipient or have received this message in error, please notify me immediately and delete this message from your computer system. Any unauthorized use or distribution is prohibited. Please consider the environment before printing this email.
_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Fri Jan 4 22:39:37 2013

This archive was generated by hypermail 2.1.8 : Mon Jan 07 2013 - 08:57:08 MST