Re: Time unlimited.

The following is from an email I sent some time ago.
It is a conceptual 'pictograph' of what data on a netCDF files look like
when written with and without an unlimited record dimension
[usually "time"] .

Note: my mailer sometime reformats my emails [frustrating!]
so I'm not sure how the alignments will look when sent.
========================

Let's say there are 2 variables a(time,lev,lat,lon), b(time,lat,lon)
where time>1

If the record dimension is not tagged as unlimited , the data are written
as a block write.

     ---------
     netCDF header stuff:
     variable/dimension names, dimension sizes, attributes,......
     ---------
     | | t=0 [lev,lat,lon]
     | | t=1
     | |
     | a |
     | |
     | |
     | | t=N
     ---------
followed by
     ---------
     | | t=0 [lat,lon]
     | | t=1
     | |
     | b |
     | |
     | |
     | | t=N
     ---------

Now, let's say u want to add new times [a' and b'] to
both a and b [say, A=a+a' and B=b+b'].
This might be monthly or daily updates. Well, you basically
have to recreate the whole file again

     --------- ---------
     | | | |
     | | | |
     | | | |
     | a | => | A |
     | | | |
     | | | |
     | | | |
     --------- | |
                                  | |
                                  | |
                                  | |
                                ---------

     --------- ---------
     | | | |
     | | | |
     | b | | |
     | | | B |
     --------- | |
                                  | |
                                  | |
                                  | |
                                  ---------

With an unlimited dimension the variables are interleaved with
the record dimension [typically, "time"]

                ________
    t=0 | |
                | a | a(lev,lat,lon) at t=0
                | |
                | |
                --------
                | |
                | b | b(lat,lon) at t=0
                --------
                ________
     ti=1 | |
                | a | a(lev,lat,lon) at t=1
                | |
                | |
                --------
                | |
                | b | b(lat,lon) at t=1
                --------
                   :
                   :
                   :
                ________
     t=N | |
                | a | a(lev,lat,lon) at t=N
                | |
                | |
                --------
                | |
                | b | b(lat,lon) at t=N
                --------

Now if you have one or more (say) monthly records to add, the data
can be appended to the current file because you are just adding
additional records at the end.

                ________
   t=N+1 | |
                | a' | a'(lev,lat,lon) at t=N+1
                | |
                | |
                --------
                | |
                | b' | b'(lat,lon) at t=N+1
                --------
                ________
   t=N+2 | |
                | a' | a'(lev,lat,lon) at t=N+2
                | |
                | |
                --------
                | |
                | b' | b'(lat,lon) at t=N+2
                --------
                   :
                  etc
                   :
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

==============================

OK ... now the key .... the netCDF operators [say, ncrcat]

Let's say u have FOO1.nc spannin (say) 1/1950-12/2007.
and you want to add a new month: create a single monthly
file [FOO2.nc] containing 1/2008 ; use

       ncrcat FOO1.nc FOO2.nc FOO_3.nc [new file]

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Dave Allured wrote:
> Yi,
>
> You mean you actually want to change each file on disk, right?
>
> Because of the fundamental structure of Netcdf 3 files, each file must
> be completely rewritten when changing a dimension from limited to
> unlimited, or reverse.
>
> The simplest and most robust method that I know does not use NCL. This
> changes the time dimension from limited to unlimited. Reverse str1
> and str2 to go the other way. Change the dim name and integer value
> to match your files. C shell syntax:
>
> set str1 = 'time = 24350 ;'
> set str2 = 'time = UNLIMITED ; // (24350 currently)'
> ncdump in.nc | sed -e "s#^.$str1# $str2#" | ncgen -o out.nc
>
> Do this in a shell script, for each input and output file name. If
> the integer dimension value changes between files, then you will need
> to expand the sed expression to accommodate a changing dimension.
>
> Dave Allured
> CU/CIRES Climate Diagnostics Center (CDC)
> http://cires.colorado.edu/science/centers/cdc/
> NOAA/ESRL/PSD, Climate Analysis Branch (CAB)
> http://www.cdc.noaa.gov/
>
> Wang, Yi wrote:
>> Hi Folks:
>>
>> Is there a simple way to change time to unlimited for a bunch of large
>> netcdf files?
>>
>> Each netcdf file has more than 20 variables, in which some have time
>> dimension, some do not have time dimension.
>>
>> I can use the " filedimdef(outfile,"time",-1,True)" to make time
>> unlimited.
>>
>> However, I need write all those variables in the script.
>>
>> Thanks,
>>
>> Yi Wang, PhD

_______________________________________________
ncl-talk mailing list
ncl-talk_at_ucar.edu
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Sat Feb 09 2008 - 11:16:07 MST