Re: reading csv data with missing values

All,

A new function str_split_csv (6.1.0) has made it easy to correctly read CSV
files with empty fields as missing values. Try this program with the test
file attached to Zhang's original message:

begin
  infile = "test-with-missing.csv"
  read_whole_file = -1
  lines = asciiread (infile, read_whole_file, "string")
  opt_quoting = 0
  strs = str_split_csv (lines, ",", opt_quoting)
  nums = tofloat (strs)
  print (strs + " " + nums)
end

CSV files with blank cells are very common in "the real world". Please
consider adding this simple demo to the NCL examples page for "Reading CSV
files". Thank you. Thanks also to the NCL team for adding str_split_csv,
very useful.

--Dave

On Wed, Feb 26, 2014 at 9:17 PM, Will Hobbs <Will.Hobbs@utas.edu.au> wrote:

> Bin
>
> The reason that NCL doesn't recognise the missing data is that those
> entries are completely blank; you need something in there or NCL won't be
> able to recognise those blank spaces as missing values.
>
> Try putting a FillValue in the blank fields; this can be any value that
> is outside the physical range of you input data, although -999.9 is the NCL
> default.
> Then, assign the FillValue as an attribute after you have read the csv
>
> >data = asciiread("data/test.csv",(/16,2/),"float")
> >data@_FillValue = -999.9 ;(or whatever number you have assigned in the
> input .csv file
>
> Hope that helps
>
> Will
>
> From: zhangbinrc <zhangbinrc@163.com>
> Date: Thursday, 27 February 2014 2:59 PM
> To: "ncl-talk@ucar.edu" <ncl-talk@ucar.edu>
> Subject: reading csv data with missing values
>
> Hi all,
>
> I¡¯m writing a script to read in a 16x2 csv file with 3 values missing. The
> main script lines are as following:
>
> data = asciiread("data/test.csv",(/16,2/),"float")
>
> print(data(:,1))
>
> The problem is that the missing values are skipped. Is there any way that
> I can keep the missing values in the right location of the matrix ?
>
> The file is attached here.
>
> Thanks!
>
> ÕÅ±ó ²©Ê¿Éú/Bin Zhang Ph.D. Candidate
> Ç廪´óѧµØÇòϵͳ¿ÆѧÑо¿ÖÐÐÄ/ Center for Earth System Science, Tsinghua
> Email:zhangbin06@mails.tsinghua.edu.cn
>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Fri Feb 28 14:53:02 2014