Fwd: csv files

Ben,

The problem is that your .csv file is generated/outputted from excel,
or other MS office software.
Which instead of have a new-line character to separate each line, it
uses carriage-return
(looks like ^M on Unix system). That is also why NCL "asciiread"
cannot read it, as carriage-return
is not a common ascii character.

You may try the attached perl script (tst.pl) to replace carriage-
return with new-line.
Use command "tst.pl your.csv" to run the program. BEWARE that
"your.csv" file will be OVERWRITTEN,
so, make a copy before you run this program.

Since your file is csv, so please change the delimiter to 'delim =
","', instead of using TAB as your delimiter.
Attached ncl script has been tested to read the Perl processed data.

Regards,

Wei
huangwei@ucar.edu
VETS/CISL
National Center for Atmospheric Research
P.O. Box 3000 (1850 Table Mesa Dr.)
Boulder, CO 80307-3000 USA
(303) 497-8924

Begin forwarded message:

> From: Benjamin Lamptey <bllamptey@gmail.com>
> Date: August 11, 2010 1:24:03 AM MDT
> To: Wei Huang <huangwei@ucar.edu>
> Subject: Re: csv files
>
> Wei,
> Here you go.
>
> Ben
>
> On Tue, Aug 10, 2010 at 7:45 PM, Wei Huang <huangwei@ucar.edu> wrote:
> Ben,
>
> Attach a data file, and I'll try.
>
> Wei Huang
> huangwei@ucar.edu
> VETS/CISL
> National Center for Atmospheric Research
> P.O. Box 3000 (1850 Table Mesa Dr.)
> Boulder, CO 80307-3000 USA
> (303) 497-8924
>
>
>
>
>
> On Aug 10, 2010, at 1:33 PM, Benjamin Lamptey wrote:
>
>> Wei,
>> The problem is the data is not read in at all. That is, I
>> do not get Accra_rr. See below for the output
>> xxxxxx
>> Variable: Accra_rr
>> Type: string
>> Total Size: 4 bytes
>> 1 values
>> Number of Dimensions: 1
>> Dimensions and sizes: [1]
>> Coordinates:
>> (0) missing
>> fatal:Subscript out of range, error in subscript #0
>> fatal:An error occurred reading Accra_rr
>> fatal:Execute: Error occurred at or near line 23 in file prelim.ncl
>> xxxxxxxxxxxx
>>
>> Regards
>> Ben
>> On Tue, Aug 10, 2010 at 2:15 PM, Wei Huang <huangwei@ucar.edu> wrote:
>> Ben,
>>
>> After you read in, Accra_rr is an array of string.
>> Where you use str_get_field to get c1, and c2, here
>> both c1 and c2 are still arrays of string.
>>
>> You can use Accra_rain = tofloat(c2) to get the rain,
>> now being converted from string to float.
>> But c1 CANNOT be converted to date simple,
>> because you have those characters, but you can use:
>>
>> str_days = str_get_field(c1, 1, "-")
>> str_months = str_get_field(c1, 2, "-")
>> str_years = str_get_field(c1, 3, "-")
>>
>> print("days: " + str_days + ", month: " + str_months + ", years: "
>> + str_years)
>>
>> But remember, where days, months, and years are still arrays of
>> string.
>>
>> You may use: days = toint(str_days) to convert str_days to integer
>> days.
>>
>> Wei
>>
>>
>> huangwei@ucar.edu
>> VETS/CISL
>> National Center for Atmospheric Research
>> P.O. Box 3000 (1850 Table Mesa Dr.)
>> Boulder, CO 80307-3000 USA
>> (303) 497-8924
>>
>>
>>
>>
>>
>> On Aug 10, 2010, at 1:54 AM, Benjamin Lamptey wrote:
>>
>>>
>>> Hi Wei,
>>> I think the issue is because the date is a mixture of string and
>>> float (or integer).
>>> When I try to read in float (see attached script), I get the
>>> following (note string part
>>> of date is not read and rain values are not treated as separated
>>> by tab);
>>> xxxxxxxxxxx
>>> Copyright (C) 1995-2010 - All Rights Reserved
>>> University Corporation for Atmospheric Research
>>> NCAR Command Language Version 5.2.1
>>> The use of this software is governed by a License Agreement.
>>> See http://www.ncl.ucar.edu/ for more details.
>>>
>>>
>>> Variable: Accra_rr
>>> Type: float
>>> Total Size: 209528 bytes
>>> 52382 values
>>> Number of Dimensions: 1
>>> Dimensions and sizes: [52382]
>>> Coordinates:
>>> (0) 1
>>> (1) -60
>>> (2) 0
>>> (3) 2
>>> (4) -60
>>> (5) 0
>>> (6) 3
>>> (7) -60
>>> (8) 0
>>> xxxxxxxxx
>>>
>>> When I try to read as string, I get the following:
>>>
>>> xxxxxxx
>>> Copyright (C) 1995-2010 - All Rights Reserved
>>> University Corporation for Atmospheric Research
>>> NCAR Command Language Version 5.2.1
>>> The use of this software is governed by a License Agreement.
>>> See http://www.ncl.ucar.edu/ for more details.
>>> warning:asciiread: No elements read from file, returning a single
>>> element with the default missing value for the requested type
>>> ^M
>>>
>>> Variable: Accra_rr
>>> Type: string
>>> Total Size: 4 bytes
>>> 1 values
>>> Number of Dimensions: 1
>>> Dimensions and sizes: [1]
>>> Coordinates:
>>> (0) missing
>>> fatal:Subscript out of range, error in subscript #0
>>> ^Mfatal:An error occurred reading Accra_rr
>>> ^Mfatal:Execute: Error occurred at or near line 23 in file
>>> prelim.ncl
>>>
>>> xxxxxxxx
>>>
>>> On the Reading csv files website, there is no example of a
>>> situation where
>>> one has a mixture of string and integer (or float) the way the
>>> dates are structured.
>>>
>>> Cheers
>>> Ben
>>> On Mon, Aug 9, 2010 at 2:56 PM, Wei Huang <huangwei@ucar.edu> wrote:
>>> Ben,
>>>
>>> In your script, change these two lines:
>>> c1 = str_get_field(infile(1:), 1, delim)
>>> c2 = str_get_field(infile(1:), 2, delim)
>>>
>>> to:
>>> c1 = str_get_field(Accra_rr(1:), 1, delim)
>>> c2 = str_get_field(Accra_rr(1:), 2, delim)
>>>
>>> Then it will work.
>>>
>>> You are suppose to process the strings, with string function,
>>> not the file.
>>>
>>> Wei
>>>
>>> huangwei@ucar.edu
>>> VETS/CISL
>>> National Center for Atmospheric Research
>>> P.O. Box 3000 (1850 Table Mesa Dr.)
>>> Boulder, CO 80307-3000 USA
>>> (303) 497-8924
>>>
>>>
>>>
>>>
>>>
>>> On Aug 9, 2010, at 2:42 AM, Benjamin Lamptey wrote:
>>>
>>>> Wei,
>>>> Good morning. This is what I get when I try your suggestion.
>>>>
>>>> xxxxxxx
>>>> Copyright (C) 1995-2010 - All Rights Reserved
>>>> University Corporation for Atmospheric Research
>>>> NCAR Command Language Version 5.2.1
>>>> The use of this software is governed by a License Agreement.
>>>> See http://www.ncl.ucar.edu/ for more details.
>>>> warning:asciiread: No elements read from file, returning a single
>>>> element with the default missing value for the requested type
>>>>
>>>>
>>>> Variable: Accra_rr
>>>> Type: string
>>>> Total Size: 4 bytes
>>>> 1 values
>>>> Number of Dimensions: 1
>>>> Dimensions and sizes: [1]
>>>> Coordinates:
>>>> (0) missing
>>>> fatal:Subscript out of range, error in subscript #0
>>>> fatal:An error occurred reading infile
>>>> fatal:Execute: Error occurred at or near line 22 in file prelim.ncl
>>>> xxxxxxxxxxxxxx
>>>>
>>>> Kindly use the attached ncl script and the data I sent in my
>>>> first email to try out your thoughts and see.
>>>>
>>>> Thanks
>>>> Ben
>>>> On Fri, Aug 6, 2010 at 4:31 PM, Wei Huang <huangwei@ucar.edu>
>>>> wrote:
>>>> Ben,
>>>>
>>>> Try this script:
>>>>
>>>> strs = asciiread("tst.txt", -1, "string")
>>>> delim = str_get_tab()
>>>> c1 = str_get_field(strs(1:), 1, delim)
>>>> c2 = str_get_field(strs(1:), 2, delim)
>>>>
>>>> print(c1)
>>>> print(c2)
>>>>
>>>> (Certainly, you need to change tst.txt to your own file name).
>>>>
>>>> Regards,
>>>>
>>>> Wei
>>>>
>>>> huangwei@ucar.edu
>>>> VETS/CISL
>>>> National Center for Atmospheric Research
>>>> P.O. Box 3000 (1850 Table Mesa Dr.)
>>>> Boulder, CO 80307-3000 USA
>>>> (303) 497-8924
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Aug 6, 2010, at 10:00 AM, Benjamin Lamptey wrote:
>>>>
>>>>> Hello,
>>>>> I have some .csv file which looks like the following
>>>>>
>>>>>
>>>>> Date RR
>>>>> 1-Jan-60 0
>>>>> 2-Jan-60 0
>>>>> 3-Jan-60 0
>>>>> 4-Jan-60 0
>>>>> 5-Jan-60 0
>>>>> 6-Jan-60 0
>>>>> 7-Jan-60 0
>>>>> 8-Jan-60 0
>>>>> 9-Jan-60 0
>>>>> 10-Jan-60 0
>>>>> 11-Jan-60 0
>>>>> 12-Jan-60 0
>>>>> 13-Jan-60 0
>>>>> 14-Jan-60 0
>>>>> 15-Jan-60 3.8
>>>>> 16-Jan-60 0
>>>>> 17-Jan-60 0
>>>>> 18-Jan-60 0
>>>>> 19-Jan-60 0
>>>>> 20-Jan-60 0
>>>>> 21-Jan-60 0
>>>>> 22-Jan-60 0
>>>>> 23-Jan-60 0
>>>>> 24-Jan-60 0
>>>>> 25-Jan-60 0
>>>>> 26-Jan-60 0
>>>>> 27-Jan-60 0
>>>>> 28-Jan-60 0
>>>>> 29-Jan-60 0
>>>>> 30-Jan-60 0
>>>>> 31-Jan-60 0
>>>>> 1-Feb-60 0
>>>>> 2-Feb-60 0
>>>>> 3-Feb-60 0
>>>>> 4-Feb-60 3.8
>>>>>
>>>>>
>>>>> I have tried using the examples on the reading csv files page
>>>>> without success.
>>>>> The date is not read in properly.
>>>>>
>>>>> Could someone kindly help me out?
>>>>>
>>>>> Thanks
>>>>> Ben
>>>>>
>>>>> Benjamin Lamptey, PhD
>>>>> Senior Lecturer
>>>>> Nautical Science Department
>>>>> Regional Maritime University
>>>>> P.O. Box GP 1115, Accra Ghana
>>>>>
>>>>> and
>>>>>
>>>>> Scientist (Meteorologist and Geoscientist)
>>>>> International Water Management Institute
>>>>> PMB CT 112, Cantonments, Accra Ghana
>>>>> FAX: +233 030 2 784752
>>>>>
>>>>> Cell: +233(0)273135062
>>>>> bllamptey@gmail.com
>>>>> http://www.rap.ucar.edu/~lamptey
>>>>>
>>>>> _______________________________________________
>>>>> ncl-talk mailing list
>>>>> List instructions, subscriber options, unsubscribe:
>>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Benjamin Lamptey, PhD
>>>> Senior Lecturer
>>>> Nautical Science Department
>>>> Regional Maritime University
>>>> P.O. Box GP 1115, Accra Ghana
>>>>
>>>> and
>>>>
>>>> Scientist (Meteorologist and Geoscientist)
>>>> International Water Management Institute
>>>> PMB CT 112, Cantonments, Accra Ghana
>>>> FAX: +233 030 2 784752
>>>>
>>>> Cell: +233(0)273135062
>>>> bllamptey@gmail.com
>>>> http://www.rap.ucar.edu/~lamptey
>>>>
>>>> <prelim.ncl>
>>>
>>>
>>>
>>>
>>> --
>>> Benjamin Lamptey, PhD
>>> Senior Lecturer
>>> Nautical Science Department
>>> Regional Maritime University
>>> P.O. Box GP 1115, Accra Ghana
>>>
>>> and
>>>
>>> Scientist (Meteorologist and Geoscientist)
>>> International Water Management Institute
>>> PMB CT 112, Cantonments, Accra Ghana
>>> FAX: +233 030 2 784752
>>>
>>> Cell: +233(0)273135062
>>> bllamptey@gmail.com
>>> http://www.rap.ucar.edu/~lamptey
>>>
>>> <prelim.ncl>
>>
>>
>>
>>
>> --
>> Benjamin Lamptey, PhD
>> Senior Lecturer
>> Nautical Science Department
>> Regional Maritime University
>> P.O. Box GP 1115, Accra Ghana
>>
>> and
>>
>> Scientist (Meteorologist and Geoscientist)
>> International Water Management Institute
>> PMB CT 112, Cantonments, Accra Ghana
>> FAX: +233 030 2 784752
>>
>> Cell: +233(0)273135062
>> bllamptey@gmail.com
>> http://www.rap.ucar.edu/~lamptey
>>
>
>
>
>
> --
> Benjamin Lamptey, PhD
> Senior Lecturer
> Nautical Science Department
> Regional Maritime University
> P.O. Box GP 1115, Accra Ghana
>
> and
>
> Scientist (Meteorologist and Geoscientist)
> International Water Management Institute
> PMB CT 112, Cantonments, Accra Ghana
> FAX: +233 030 2 784752
>
> Cell: +233(0)273135062
> bllamptey@gmail.com
> http://www.rap.ucar.edu/~lamptey
>

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Wed Aug 11 08:26:22 2010