Re: reading all the variables from a netCDF when the variable names are unknown (or too many)

Hi Jonathan,

As a follow up of my previous mail related to "eval", why don't you
run ncl on your code twice? The first pass will create NCL code that
will be run in the second pass.

Here is my crude semi-pseudo-code

Start of NCL file ----------

_NCLCode="" ; declared at the outset. global variable??

function cr()
begin
  return(inttochar(10))
end

procedure add_msg(msg)
begin
  _NCLCode=_NCLCode+cr+msg
end

begin
  do ivar = 0,nvar-1
    msg =varnames(ivar)+" = fin->$"+varnames(ivar)+"$"
    add_msg(msg)
  end do

  asciiwrite("runit.ncl",_NCLCode)
  system("ncl runit.ncl")
end

* Jonathan Vigh <vigh_at_atmos.colostate.edu> [2009-02-24 21:05:59 +0000]:

>
> Hi David,
> I'll likely be doing some conditional processing, where some
> variables affect others, so I'd like to have them all in memory at once.
> For this specific application, all the variables are quite small - the
> entire netCDF file is <50 KB - so it won't be a problem to hold them all
> in memory. I'm planning to read in about 200 of these files and
> aggregate some of the 177 variables into several multidimensional arrays
> for further data analysis. The variables are a hodgepodge of arrays of
> strings, logicals, integers, floats, etc. It's a mess. I've never had to
> work with such a disparate collection of data before. The netCDF files
> I'm reading are actually an intermediate (but necessary) step - the data
> were originally read from several very nonstandard ascii file formats
> and I wrote them to the netCDF files. I wrote out the 177 variables
> using a general method, so that's why I was hoping to read in the 177
> variables in a general way. The trouble with creating the variable
> without knowing it's name seems to be the main hitch at the moment.
> Accessing it without knowing it's name is the other hitch.
>
> So basically, I think need a "create_new_var" procedure (not function)
> which would act like:
> create_new_var( varname, vartype, vardimsize )
>
> One the variable is created in memory, I'd need a way to reference it in
> order to assign data to it from the file.
>
> So my code to read in everything would be something like:
> varnames = getfilevarnames(fin) ; Returns an array of file
> variable names in the file
> vartypes = getfilevartypes(fin,varnames) ; Returns the type of
> each variable name listed. A missing value is returned for any variable
> name that doesn't exist in the file.
>
> nvars = dimsizes(varnames)
> do ivar = 0, nvars-1
> create_new_var( varnames(ivar), vartypes(ivar),
> getfilevardimsizes(fin,varnames(ivar)) )
> #varnames(ivar)# = fin->$varnames(ivar)$ ; where # # is used
> to indicate the way to reference the variable in memory using the string
> value from my array varnames that was queried from the file
> end do
>
> If there was a way to reference the variable from memory, one could do
> all the processing on an arbitrary variable without ever knowing it's
> name. This would be a nice feature to have.
>
> Jonathan
>
>
>
> Dave Allured wrote:
> > Jonathan,
> >
> > Do you need to have all the variables in memory at the same time, or
> > can you just process them one at a time from the input file to the
> > output file(s)? The second has a much easier answer than the first.
> > A little more information about the nature of your application is
> > needed.
> >
> > Dave Allured
> > CU/CIRES Climate Diagnostics Center (CDC)
> > http://cires.colorado.edu/science/centers/cdc/
> > NOAA/ESRL/PSD, Climate Analysis Branch (CAB)
> > http://www.cdc.noaa.gov/psd1/
> >
> > Jonathan Vigh wrote:
> >
> >> Greetings NCL'ers,
> >> I'd like to read in all the variables from a netCDF file in a
> >> general way because there are 177 of them. I understand how to query the
> >> file and get a list of variable names, and I even understand how to
> >> reference the variable using:
> >> fin->$varnames$
> >>
> >> What I am confused about is what to assign this to. I'd like to avoid
> >> having to type out 177 lines to read in each variable if possible. But
> >> if I didn't even know the variable names in advance, I'd still like to
> >> know how to do this.
> >>
> >> Maybe I've missed something really basic, but I couldn't seem to find
> >> anything from the documentation. If anyone has an idea, I'd be most
> >> grateful. It would be nice if NCL could simply make all the variables in
> >> a netCDF file available without having to go through this step, so maybe
> >> this could be the prototype for a new function that does this.
> >>
> >> Thanks,
> >> Jonathan
> >>
> >>
> >> begin
> >> processed_filename = systemfunc("cd " + processed_data_directory + ";
> >> ls " + stormid + "*.nc")
> >> fin = addfile(processed_data_directory + processed_filename,"r") ;
> >> open output netCDF file
> >>
> >> ; query the file to see what variables it contains and what the sizes of
> >> the file dimensions are
> >> varnames = getfilevarnames(fin) ; Returns an array of file
> >> variable names in the file
> >> filedimsizes = getfiledimsizes(fin) ; Returns a list of the
> >> sizes of all the dimensions in the file.
> >>
> >> ; query all the named variables in the file
> >> vartypes = getfilevartypes(fin,varnames) ; Returns the type of
> >> each variable name listed. A missing value is returned for any variable
> >> name that doesn't exist in the file.
> >>
> >>
> >> ; loop through all the variables, query them, and read them in
> >> nvars = dimsizes(varnames)
> >> do ivar = 0, nvars-1
> >> print("Reading variable ivar = "+ivar+" "+vartypes(ivar)+"
> >> "+varnames(ivar))
> >> ???? = fin->$varnames(ivar)$ ; I understand how to
> >> read in a variable from a file if I don't know the variable name in
> >> advance, but how can I assign it to a variable without typing the
> >> variable's name?
> >> end do
> >>
> >> end
> >> _______________________________________________
> >> ncl-talk mailing list
> >> List instructions, subscriber options, unsubscribe:
> >> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
> >>
> >
> > _______________________________________________
> > ncl-talk mailing list
> > List instructions, subscriber options, unsubscribe:
> > http://mailman.ucar.edu/mailman/listinfo/ncl-talk
> >
>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>

-- 
Saji N. Hameed
APEC Climate Center          				+82 51 668 7470
National Pension Corporation Busan Building 12F         
Yeonsan 2-dong, Yeonje-gu, BUSAN 611705			saji_at_apcc21.net
KOREA
_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk