Re: Re: [ncl-talk] How to check and fill the missing lines with the missing values?

Hi, Mary

Thanks a lot for your help. Yes, I am reading a lot of ascii files. Below is an example of the file, there is no "..." in the file, and the missing days are just omitted, without blank lines.

I want to find these missing days and fill the lines with missing values for the varibles. Could you please give me some suggetions about this? Thanks a lot.

STN--- WBAN YEARMODA TEMP DEWP SLP STP VISIB WDSP MXSPD GUST MAX MIN PRCP SNDP FRSHTT
106160 99999 19730101 27.1 24 13.4 24 9999.9 0 965.9 4 6.2 24 2.7 24 6.0 999.9 35.6* 19.4* 0.00I 999.9 000000
106160 99999 19730102 27.9 24 16.4 24 9999.9 0 9999.9 0 6.0 24 2.2 24 8.9 999.9 35.6* 23.0* 0.00I 999.9 000000
106160 99999 19730107 30.4 24 28.4 24 9999.9 0 974.4 4 2.4 24 4.5 24 8.0 999.9 37.4* 26.6* 99.99 999.9 110000
106160 99999 19730108 28.5 24 27.4 24 9999.9 0 973.6 7 1.9 24 4.8 24 8.0 999.9 30.2* 28.4* 99.99 999.9 110000
106160 99999 19730109 30.4 24 29.4 24 9999.9 0 971.1 7 0.4 24 1.9 24 6.0 999.9 32.0* 28.4* 99.99 999.9 111000
106160 99999 19730117 29.7 24 27.9 24 9999.9 0 946.9 7 1.8 24 6.7 24 13.0 999.9 30.2* 28.4* 0.00I 999.9 100000
106160 99999 19730118 28.4 24 26.4 24 9999.9 0 9999.9 0 2.5 24 3.7 24 7.0 13.0 9999.9 9999.9 99.99 999.9 111000
106160 99999 19730119 27.7 24 25.3 24 9999.9 0 9999.9 0 1.4 24 1.9 24 3.9 999.9 28.4* 26.6* 99.99 999.9 111000
106160 99999 19730120 30.9 24 29.4 24 9999.9 0 947.4 7 2.1 24 6.0 24 12.0 999.9 35.6* 28.4* 99.99 999.9 111000
106160 99999 19730201 31.0 24 27.8 24 9999.9 0 9999.9 0 5.6 24 1.8 24 4.1 999.9 35.6* 28.4* 0.00I 999.9 100000
106160 99999 19730202 30.9 24 28.9 24 9999.9 0 967.3 6 2.6 24 3.6 24 8.0 999.9 32.0* 28.4* 0.00I 999.9 100000


Following is the scritpt I used to read the file.

;Read data into a big 1D string array
  fname = "data/gsod/1973/726055-99999-1973.op"
  data = asciiread(fname,-1,"string")

; Count the number of fields, just to show it can be done.
  nfields = str_fields_count(data(0)," ")
  print("number of fields = " + nfields)

;
; Skip first row of "data" because it's just a header line.
;
; Use a space (" ") as a delimiter in str_get_field. The first
; field is field=1 (unlike str_get_cols, in which the first column
; is column=0).
;
  stn = stringtoint(str_get_field(data(1::), 1," "))
  wban = stringtoint(str_get_field(data(1::), 2," "))

  yearmoda = stringtoint(str_get_field(data(1::), 3," "))
  year = stringtoint(str_get_cols(data(1::),14,17))
  month = stringtoint(str_get_cols(data(1::),18,19))
  day = stringtoint(str_get_cols(data(1::),20,21))

  temp = stringtofloat(str_get_field(data(1::), 4," "))
; Convert temperature from Fahrenheit to Celsius.
  temp = (temp-32)*5/9
  dewp = stringtofloat(str_get_field(data(1::), 6," "))
; Convert dew point temperature from Fahrenheit to Celsius.
  dewp = (dewp-32)*5/9

; Calculate relative humidity (%).
  rh = 100*(((112-0.1*temp+dewp)/(112+0.9*temp))^8)

  slp = stringtofloat(str_get_field(data(1::), 8," "))
  stp = stringtofloat(str_get_field(data(1::), 10," "))

  visib = stringtofloat(str_get_field(data(1::), 12," "))

  wdsp = stringtofloat(str_get_field(data(1::), 14," "))
  maxspd = stringtofloat(str_get_field(data(1::), 16," "))
  gust = stringtofloat(str_get_field(data(1::), 17," "))

  maxtemp = stringtofloat(str_get_field(data(1::), 18," "))
; Convert maximum temperature from Fahrenheit to Celsius.
  maxtemp = (maxtemp-32)*5/9
  mintemp = stringtofloat(str_get_field(data(1::), 19," "))
; Convert minimum temperature from Fahrenheit to Celsius.
  mintemp = (mintemp-32)*5/9

  prcp = stringtofloat(str_get_field(data(1::), 20," "))
  sndp = stringtofloat(str_get_field(data(1::), 21," "))

  frshtt = stringtoint(str_get_field(data(1::), 22," "))
  fog = stringtoint(str_get_cols(data(1::),132,132))
  rain = stringtoint(str_get_cols(data(1::),133,133))
  snow = stringtoint(str_get_cols(data(1::),134,134))
  hail = stringtoint(str_get_cols(data(1::),135,135))
  thunder = stringtoint(str_get_cols(data(1::),136,136))
  tornado = stringtoint(str_get_cols(data(1::),137,137))

  print(rh)
  print(maxtemp)
  print(frshtt)
  print(yearmoda)
  print(tornado)






Wen.J.Qu
2012-08-24



发件人： Mary Haley
发送时间： 2012-08-23 14:04:55
收件人： Wen.J.Qu@gmail.com
抄送：
主题： Re: [ncl-talk] How to check and fill the missing lines with the missing values?

Shawn,

I need more information. Are these lines of data in an ascii file?

If so, does the file actually look like that, with the "…" characters, or are the lines just blank, or something else?

If these lines of data are in an ascii file, how are you reading in the file?

If the lines are just blank, then you can read the file in as strings check for blank strings using "str_is_blank".

http://www.ncl.ucar.edu/Document/Functions/Built-in/str_is_blank.shtml

--Mary

On Aug 23, 2012, at 11:09 AM, Wen.J.Qu wrote:

> Hello,
>
> I am dealing with a daily time series of multiVaribles. My problem is that there are some missing days (lines) of the data, like below
>
> 19980101 ...
> 19980102 ...
> 19980103 ...
> 19980109 ...
> 19980201 ...
> 19980218 ...
> ... ...
>
> How can I check and fill these missing lines (days) with the missing values?
>
> Thanks a lot.
>
>
> Shawn
>
> Wen.J.Qu
> 2012-08-23
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Fri Aug 24 08:57:26 2012