Re: TRMM 3B42 V7 HDF files

Dave Adams,

Now we are getting somewhere. Now the likely problem is that the addfile
command on original line 233 will not overwrite an existing netcdf file.
 This is for the user's protection. Try moving or deleting the previous
output file by the same name, and see if that helps.

You could also enable the "rm" command on original line 229, which deletes
a previous file by the same name. I consider this dangerous when working
with large data sets, but it's a common practice. You could enable "rm"
for debugging, then disable it for your final version to keep.

If this is not the right problem, then you might consider upgrading your
NCL version. 6.0.0 is over three years old. There have been many many bug
fixes in the Netcdf support libraries, especially in Netcdf-4 format
support which is used by this version.

I hope you are on the right track now. I am done for tonight.

--Dave Allured


On Mon, Jun 9, 2014 at 8:23 PM, David Adams <dave.k.adams@gmail.com> wrote:

> Hi Dave,
> first problem found
> no " " around load substring.ncl. I have added them.
>
> I changed "find " command to file my uncompressed files.
> find_command = "find -name '3B42.*.HDF'"
>
> It finds the right number of files 2920, but I get this error:
> Seems to be the output file option.
>
>
> NCAR Command Language Version 6.0.0
> The use of this software is governed by a License Agreement.
> See http://www.ncl.ucar.edu/ for more details.
> (0)
> (0) Convert TRMM three hourly HDF files into single Netcdf file.
> (0) Start: Mon Jun 9 21:18:28 CDT 2014
> (0)
> (0) Input base directory = /home/david/Desktop/TRMM_Data/2013
> (0) First input file = ./3B42.20130525.00.7.HDF
> (0) Last input file = ./3B42.20130408.18.7.HDF
> (0) Number of input files = 2920
> (0)
> (0) Begin output file:
> (0) /home/david/Desktop/TRMM_Data/2013/
> precip.trmm.8x.v7a.1998-2013.nc
> (0) Output format = NetCDF4Classic
> fatal:Could not create (/home/david/Desktop/TRMM_Data/2013/
> precip.trmm.8x.v7a.1998-2013.nc)
> warning:FileSetFileOption: invalid file or format
> (0) Write metadata and coordinate variables.
> fatal:Execute: Error occurred at or near line 247 in file
> hdf-to-nc.trmm.3b42.8x.v7a.ncl
>
> thanks,
> Dave
>
>
>
>
>
>
> On Mon, Jun 9, 2014 at 8:57 PM, Dave Allured - NOAA Affiliate <
> dave.allured@noaa.gov> wrote:
>
>> Dave Adams,
>>
>> It seems that the main program is not even starting to run properly. The
>> words "substring.ncl" and "begin" are in the output that you showed. These
>> words should not be printed, they are part of the program.
>>
>> Also, the very first message that you should see after the NCL header is
>> "Convert TRMM three hourly HDF files into single Netcdf file". This
>> message was NOT in the output that you showed. Therefore, the program is
>> not starting properly.
>>
>> I do not understand what is going on. Let me ask a few really basic
>> questions in case you are just getting started.
>>
>> Did you get the second attachment "substring.ncl" and put it into the
>> same directory as the main program?
>>
>> Did you save both NCL attachments using your mail reader's save function?
>> Did the files get corrupted while saving?
>>
>> Are you starting this program on an operating system (Linux, etc.)
>> command line with "ncl" followed by the name of the main program, as shown
>> on line 100 in the program?
>>
>> Try checking syntax of the "load" commands with the NCL documentation.
>> There must be double quotes around the file name.
>>
>> Try making a test copy of the main program. Delete all lines from "Data
>> set parameters" (line 112) to the end of the program, except for "end" at
>> the very bottom. The test program should now consist only of load and
>> print statements, and comments, except for original line 29 which can also
>> be deleted.
>>
>> You can also delete all the comments for the sake of debugging. Also
>> delete the load commands themselves if nothing else works.
>>
>> See if you can get this simple test of a few print statements to work.
>> Good luck.
>>
>> --Dave Allured
>>
>>
>> On Mon, Jun 9, 2014 at 6:51 PM, David Adams <dave.k.adams@gmail.com>
>> wrote:
>>
>>> Hi Dave et al.
>>>
>>> I have set it to
>>> ./substring.ncl
>>> in hdf-to-nc.trmm.3b42.8x.v7a.ncl\
>>>
>>> but still get the same error.
>>>
>>> See http://www.ncl.ucar.edu/ for more details.
>>> ./substring.ncl
>>> begin
>>> print(fatal:Could not open ()
>>> fatal:error at line 107 in file hdf-to-nc.trmm.3b42.8x.v7a.ncl
>>>
>>> saludos,
>>> Dave
>>>
>>>
>>> On Mon, Jun 9, 2014 at 7:45 PM, Dave Allured - NOAA Affiliate <
>>> dave.allured@noaa.gov> wrote:
>>>
>>>> Dave Adams,
>>>>
>>>> Please, no private replies. Include the mailing list in all replies.
>>>>
>>>> In the main program, remove the prefix $LIB_NCL/ from the front of the
>>>> load command near the top of the program. Or else, in the shell before
>>>> running the program, set an environment variable LIB_NCL to "." (single
>>>> period).
>>>>
>>>> --Dave Allured
>>>>
>>>>
>>>> On Mon, Jun 9, 2014 at 6:08 PM, David Adams <dave.k.adams@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Dave,
>>>>> I am trying your code with just 2013 data. The files are complete for
>>>>> 2013 uncompressed HDF files. I am getting an error
>>>>>
>>>>> University Corporation for Atmospheric Research
>>>>> NCAR Command Language Version 6.0.0
>>>>> The use of this software is governed by a License Agreement.
>>>>> See http://www.ncl.ucar.edu/ for more details.
>>>>> substring.ncl
>>>>> begin
>>>>> print(fatal:Could not open ()
>>>>> fatal:error at line 107 in file hdf-to-nc.trmm.3b42.8x.v7a.ncl
>>>>>
>>>>> Is there something I am supposed to set for the substring.ncl program?
>>>>>
>>>>> saludos,
>>>>> Dave
>>>>>
>>>>>
>>>>> On Mon, Jun 9, 2014 at 3:08 PM, Dave Allured - NOAA Affiliate <
>>>>> dave.allured@noaa.gov> wrote:
>>>>>
>>>>>> Dave Adams,
>>>>>>
>>>>>> Attached is an NCL script to translate original TRMM 3B42 V7 HDF
>>>>>> files and aggregate them into a single large Netcdf file. This approach
>>>>>> seemed to be more efficient than working with Netcdf files translated by
>>>>>> the Mirador server. You might find this useful.
>>>>>>
>>>>>> --Dave Allured
>>>>>>
>>>>>>
>>>>>> On Mon, Jun 9, 2014 at 9:11 AM, David Adams <dave.k.adams@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi NCLers
>>>>>>>
>>>>>>> Is there an easy way to convert HDF files to netcdf files for the
>>>>>>> TRMM
>>>>>>> 3B42 V7 data? I had done it before with v6.
>>>>>>>
>>>>>>> ncl_convert2nc 3B42.20121230.12.7.HDF
>>>>>>> doesn't seem to work.
>>>>>>> I get the following error
>>>>>>>
>>>>>>> warning:HDF: fakeDim2 is a zero length dimension some variables may
>>>>>>> be ignored
>>>>>>> warning:HDF: fakeDim3 is a zero length dimension some variables may
>>>>>>> be ignored
>>>>>>> warning:HDF: fakeDim4 is a zero length dimension some variables may
>>>>>>> be ignored
>>>>>>> warning:HDF: A zero length dimension was found in variable
>>>>>>> (InputFileNames), ignoring this variable
>>>>>>> warning:HDF: A zero length dimension was found in variable
>>>>>>> (InputAlgorithmVersions), ignoring this variable
>>>>>>> warning:HDF: A zero length dimension was found in variable
>>>>>>> (InputGenerationDateTimes), ignoring this variable
>>>>>>> ncdimdef: ncid 65536: NetCDF: NC_UNLIMITED size already in use
>>>>>>> ncdimdef: ncid 65536: NetCDF: NC_UNLIMITED size already in use
>>>>>>> fatal:Execute: Error occurred at or near line 360 in file
>>>>>>> /tmp/tmp5008.ncl
>>>>>>>
>>>>>>> Or are there 3B42 V7 in netcdf format for the entire data set that I
>>>>>>> can download with wget?
>>>>>>>
>>>>>>> thanks,
>>>>>>> Dave
>>>>>>>
>>>>>>
>

Received on Tue Jun 10 02:45:16 2014