errors in using 'addfiles' ( ?? )

From: Li, Yongzuo <yongzuo.li_at_nyahnyahspammersnyahnyah>
Date: Sat Jul 21 2012 - 16:12:00 MDT

Hai Wang,

Can you add print(dimsizes(fils)) before f=addfiles?
f = addfiles(fils(0),"r")
f = addfiles(fils(1),"r")

Yongzuo
________________________________________
From: ncl-talk-bounces@ucar.edu [ncl-talk-bounces@ucar.edu] on behalf of ncl-talk-request@ucar.edu [ncl-talk-request@ucar.edu]
Sent: Saturday, July 21, 2012 1:00 PM
To: ncl-talk@ucar.edu
Subject: ncl-talk Digest, Vol 104, Issue 42

Send ncl-talk mailing list submissions to
        ncl-talk@ucar.edu

To subscribe or unsubscribe via the World Wide Web, visit
        http://mailman.ucar.edu/mailman/listinfo/ncl-talk
or, via email, send a message with subject or body 'help' to
        ncl-talk-request@ucar.edu

You can reach the person managing the list at
        ncl-talk-owner@ucar.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of ncl-talk digest..."

Today's Topics:

   1. Re: Boxplot resources (Noel Aloysius)
   2. NCL documentation (Noel Aloysius)
   3. Re: NCL documentation (Mateus Teixeira)
   4. Re: how to read netcdf automatically (Jonathan Vigh)
   5. errors in using 'addfiles' ( ?? )

----------------------------------------------------------------------

Message: 1
Date: Fri, 20 Jul 2012 16:26:24 -0400
From: Noel Aloysius <noel.aloysius@gmail.com>
Subject: Re: Boxplot resources
To: NCL Talk <ncl-talk@ucar.edu>
Message-ID:
        <CANfXz8HOv3ZSafKFStcsL3rDnXmQiXZrEd1wuFF=+f03JoPwyw@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

Thank Adam. I managed to get what I want with gsn_add_polygon.

Noel

On Fri, Jul 20, 2012 at 12:31 PM, Adam Phillips <asphilli@ucar.edu> wrote:

> Hi Noel,
> The boxes created by boxplot are drawn by multiple calls to gsn_polyline.
> Thus, there is no way to automatically fill the boxes. (This could be a
> possible upgrade in the future, however the function would need to be
> completely rewritten to do this.)
>
> To draw boxes around the boxes, you can call gsn_add_polygon:
> http://www.ncl.ucar.edu/Document/Graphics/Interfaces/gsn_add_polygon.shtml
> Note that you need to call draw and frame after you call gsn_add_polygon.
> So the order of calling would be:
> boxplot
> gsn_add_polygon
> draw
> frame
>
> Good luck,
> Adam
>
>
> On 7/20/12 9:46 AM, Noel Aloysius wrote:
>
> Hi NCL,
>
> I created a box plot comparing annual and seasonal changes in
> precipitation projections (plot attached).
>
> I want to know if I can fill the boxes red and green. What is the plot
> resource that does this?
>
> I also want to draw a box around {black and red} and {blue and green}
> boxes, so that they are grouped together for increased emphasis.
>
> The relevant code section is attached.
>
> Thanks in advance for your help,
>
>
> ;***************************************************************************
>
> patSeasons = (/0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19/)
> wks = gsn_open_wks(type,diro + "pr_change_2150_boxplot") ;
> create postscript file
>
> ;------------ resources for plot background
> res =
> True ; plot mods
> desired
> res@tmXBLabels = (/" ","Ann"," ","DJF"," ","MAM"," ","JJA"," ","SON",\
> " ","Ann"," ","DJF"," ","MAM"," ","JJA","
> ","SON"/) ; labels for each box, Jan-Dec
> res@tmXBLabelFontHeightF = 0.01
> res@trYMinF = -25.0
> res@trYMaxF = 25.0
> res@vpHeightF = 0.3
> res@vpWidthF = 0.7
>
> ;------------ resources for polylines that draws the boxes
> llres = True
> llres@gsLineThicknessF = 2.5 ; line thickness
>
> ;------------ resources that control color and width of boxes
> opti = True
> opti@boxWidth = .8 ; Width of box (x
> opti@boxColors =
> (/"black","red","black","red","black","red","black","red","black","red",\
>
> "blue","green","blue","green","blue","green","blue","green","blue","green"/)
>
> plot = boxplot(wks,patSeasons,patArrSeason,opti,res,llres)
>
> draw(wks)
> frame(wks)
>
>
> ;*****************************************************************************
>
>
>
>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20120720/90dfd8a7/attachment.html

------------------------------

Message: 2
Date: Fri, 20 Jul 2012 16:29:17 -0400
From: Noel Aloysius <noel.aloysius@gmail.com>
Subject: NCL documentation
To: NCL Talk <ncl-talk@ucar.edu>
Message-ID:
        <CANfXz8Hft9MM_9pWBuHXbKz3Qv2bwFeP=tDzYvM3vmAE7GdxAg@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

Hi NCL,

I want to know if the entire ncl documentation can be downloaded and saved
as a searchable html archive. I encounter situations where I do not have
access to the internet while writing my ncl scripts.

Thanks,
Noel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20120720/3b7ed99d/attachment.html

------------------------------

Message: 3
Date: Fri, 20 Jul 2012 20:57:57 -0300
From: Mateus Teixeira <mateus.teixeira@gmail.com>
Subject: Re: NCL documentation
To: Noel Aloysius <noel.aloysius@gmail.com>
Cc: NCL Talk <ncl-talk@ucar.edu>
Message-ID:
        <CAB74pKu58ORC_kgyD=49WrML0+B9N69Y3bLY82m=uq9FqVEHqg@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

Dear Noel,

You can download NCL website with wget. Take a look at
http://www.linuxjournal.com/content/downloading-entire-web-site-wget

This linux command can copy the entire structure for a website work in a
local computer.

Best regards,

Mateus

2012/7/20 Noel Aloysius <noel.aloysius@gmail.com>

> Hi NCL,
>
> I want to know if the entire ncl documentation can be downloaded and saved
> as a searchable html archive. I encounter situations where I do not have
> access to the internet while writing my ncl scripts.
>
> Thanks,
> Noel
>
>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20120720/e8d87351/attachment.html

------------------------------

Message: 4
Date: Sat, 21 Jul 2012 00:47:26 +0000
From: Jonathan Vigh <jvigh@ucar.edu>
Subject: Re: how to read netcdf automatically
To: jcchen@gps.caltech.edu
Cc: ncl-talk@ucar.edu
Message-ID: <5009FC1E.8040809@ucar.edu>
Content-Type: text/plain; charset="iso-8859-1"

Hi Jinqiang,

   I've had this problem in the past, in which I need to read in a
netcdf file that may contain hundreds of disparate variables. The
solution of querying the file to get all the variables, then reading
them in, as Wei suggests is certainly a good way to go, but there's a
clever way to bypass this task altogether -- I've written a module that
will automatically read the contents of everything in the netcdf file
into memory. The module works by dynamically generating additional
modules and then executing them.

   Please see the attached:
module_netcdf_reader_multiple_prefix_v1.0.ncl to see how it works. To
use this module, simply load it before your main program block and set
the input filename and path, as I've illustrated in the attached script:
example_main.ncl. Then run the example_main.ncl and you should be able
to start using the variables which were read right away in the main
program block.

   The module can also read all the contents of /multiple/ netcdf files
into memory, which is double nice. If your files contain variables with
the same name, there will be problems however. to get around this, I've
written a separate module which prefixes a key to each variable name
that can be set based on a pattern in the filename. Simply swap the load
statement to load: module_netcdf_reader_multiple_prefix_v1.0.ncl instead
and then modify the part of the module where the keys are set based on
the file patterns.

   A caveat -- I believe the variables may be global when read this way.
This can be an advantage or a disadvantage, depending on your subsequent
program structure.

   It may be possible to read the input filenames and paths as Command
Line Arguments if you don't know these in advance, but I haven't tested
this yet to know if it would actually work.

   I've only tested these modules for my own codes and data. I'll be
interested to hear if other people find them useful. Of course, I can't
promise to provide any support for these. Eventually, when and if NCL
has an "eval" statement, this telescoped dynamic module approach won't
be necessary.

Best regards,
     Jonathan

On 07/20/2012 04:33 PM, Jinqiang Chen wrote:
>
> Hi there,
>
> I want to know how to read netcdf variable automatically. Now I am
> analyzing tons of CMIP5 data and I write an array of all the variable
> names and I want to use the loop to let ncl to read the specific variable
> name but I don't know how to do that. For example,
> I build a vector named variable s.t. variable = (/"cct","ccb"/)
> but it doesn't work for the case that: data = f->variable(i)
> in which i is the loop index and f is the addfile netcdf variable. Every
> time ncl read f->variable(i) it never change variable(i) to its real name,
> like cct, or ccb. Could you tell me how to deal with that?
>
> Thanks!
>
> Jinqiang Chen
>
>
> 

--
Jonathan Vigh
Project Scientist I, Joint Numerical Testbed
Research Applications Laboratory (RAL)
National Center for Atmospheric Research (NCAR)
P.O. Box 3000            tel: +1 303 497 8205
Boulder, CO 80307-3000   fax: +1 (303) 497-8171
http://www.ral.ucar.edu/staff/jvigh/
http://www.ral.ucar.edu/hurricanes/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20120721/9bd203bf/attachment.html
-------------- next part --------------
;********************************************************
; example_main.ncl
;
; This program illustrates how to use Jonathan Vigh's
; automatic netcdf reader module to read in the contents
; of multiple netcdf files into memory.
;
; Author: Jonathan Vigh (NCAR/RAL/JNT)
;
; This code was written with support from the Development Testbed Center for the NOAA HWRF Project
; 20 July 2012
;
;********************************************************
load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"
load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"
load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"
load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/shea_util.ncl"
;load "./module_climatology_parameters.ncl"
;load "./module_thermodynamics_routines.ncl"
;load "./module_model_output_plotting_routines.ncl"
;load "./module_attribute_array_routines.ncl"
;load "./module_string_routines.ncl"
;load "./module_time_routines.ncl"
;load "./module_fetch_data.ncl"
; example of how to set the filename and path to read just one file
;input_filenames  = (/"your_netcdf_filename.nc"/)
;input_filepaths  = (/"/home/yourname/path/")
; example of how to set the filenames and paths for multiple files
input_filenames  = (/"filename_1.nc4","filename_2.nc4"/)
input_filepaths  = (/"/pathtofile1/","/pathtofile2/"/)
load "./module_netcdf_reader_multiple_no_prefix_v1.0.ncl"  ; use this one to read multiple or single netcdf files with unique variable names
;load "./module_netcdf_reader_multiple_prefix_v1.0.ncl"     ; use this one if you want to have a prefix appended to all the variables names
                                                            ; based on which file they were read from -- this is necessary if your files contain
                                                            ; variables with the same names.
;********************************************************
begin
; see what was automatically read
print("The following variables have been automatically read into memory:")
list_vars()
end
-------------- next part --------------
;********************************************************
; module_netcdf_reader_multiple_no_prefix_v1.0.ncl
;********************************************************
;
; This module was written by Jonathan Vigh at Colorado State University,
; then modified for inclusion into the large-scale climatology project
; for the Development Testbed Center.
;
; IMPORTANT DISLCAIMER:
;
; All rights reserved. Absolutely no warranties are implied.
; Use this software at your own risk. I've only tested these
; modules for my own codes and data. I can not promise any
; support for these codes. Feedback is welcome: jvigh@ucar.edu.
;
; This module reads all the variables into memory from
; one or multiple netCDF files whose names have been
; defined in an array before this module is called.
; The files should be regular netcdf files.
;
; This module dynamically generates additional modules
; and then runs them to do the work necessary to bring the
; variables back into memory in the form that they were
; in before they were written out. This code does NOT
; handle logicals, strings, and character arrays with
; non-custom _FillValues (use the RS routines if the
; files contain these types).
;
; In order to use this module to read your files, you must
; do the following:
;
;    1. Load module_netcdf_interface_v1.0.ncl in your
;       script after you've loaded gsn_code.ncl, gsn_csm.ncl
;       contributed.ncl, shea_util.ncl, and any other modules
;       containing dependcies for your script. Make sure that
;       you load module_netcdf_helper_routines_v1.0.ncl BEFORE
;       you load this module, however. Here's an example of
;       the proper loading sequence:
;
;load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"
;load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"
;load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"
;load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/shea_util.ncl"
;load "./your_other_modules.ncl"
;load "./module_netcdf_interface_v1.0.ncl"
;    2. Define the following three arrays with file information:
;         input_filenames - a list of netcdf files to be read in
;         input_filepaths - a list of the paths to these files
;    3. Load this module before the begin statement of your
;       main program block.
; Now when you the script that loaded this module, all the
; variables contained in your netCDF files will be loaded
; into memory.
;
; Author: Jonathan Vigh, Joint Numerical Testbed, Research Applications Laboratory, NCAR
; Date:   20 July 2012
;
; Modification history:
;    09/20/2009         Original version written at Coloradto State University.
;    06/17/2012         Made modifications so that this can be used for the climatology project.
;                       Changed to use the normal netcdf helper routines (not the RS ones).
;
;
;
;********************************************************
; Note that we do not put the contents of this module within 'begin' and 'end' statements.
; This means that this everything in this module will be global (accessible to all other
; functions and procedures in subsequent modules and the main program which loads this
; module.
  cr          = inttochar(10)
  doublequote = inttochar(34)
  if ( (dimsizes(input_filenames) .eq. 1) .and. (dimsizes(input_filepaths) .eq. 1) ) then
;===================================================================
; Read ONE netCDF file
;===================================================================
     input_filename = input_filenames
     input_filepath = input_filepaths
     print("")
     print("Now opening the input data file: "+input_filename)
     fin = addfile(input_filepath + input_filename,"r")   ; open output netCDF file
; retrieve the global attributes from the file
     fatt_names = getvaratts(fin)
     if(.not.all(ismissing(fatt_names))) then
        print("")
        print("The file's global attributes are: ")
        print("")
        do i = 0,dimsizes(fatt_names)-1
           print(fatt_names(i)+" = "+fin@$fatt_names(i)$)
        end do
     end if
; query the file to see what variables it contains and what the sizes of the file dimensions are
     varnames = getfilevarnames(fin)            ; Returns an array of file variable names in the file
     filedimsizes = getfiledimsizes(fin)        ; Returns a list of the sizes of all the dimensions in the file.
     print("")
     print("The file's variables are:")
     print("")
;     print(varnames)
; query all the named variables in the file
     vartypes = getfilevartypes(fin,varnames)           ; Returns the type of each variable name listed. A missing value is returned for any variable name that doesn't exist in the file.
     nvars = dimsizes(varnames)
;=====================================================================================================
; Begin dynamic code generation section to read in the data from the file
     module_filename = "dynamic_module_read_netcdf.ncl"
     module_contents = ""
     do ivar = 0, nvars-1
        module_contents = module_contents + cr + "  " + varnames(ivar)+" = fin->"+varnames(ivar)
     end do
; Now write the module contents  out to a file
     if(isfilepresent(module_filename))
       system("/bin/rm -f " + module_filename)     ; remove a pre-existing file
     end if
     asciiwrite(module_filename,module_contents)
; End dynamic code generation section
;=====================================================================================================
     loadscript(module_filename)
     system("/bin/rm -f " + module_filename)     ; remove it after is has been used
  else
;===================================================================
; Read MULTIPLE netCDF files
;===================================================================
     nfiles = dimsizes(input_filenames)
;=====================================================================================================
; Begin dynamic code generation section for the outer module
     outer_module_filename = "dynamic_outer_module_read_multiple_netcdf_files.ncl"
     outer_module_contents = ""
     print_blank_line = "  print(" + doublequote + doublequote + ")"
     outer_module_contents = outer_module_contents + "  cr           = inttochar(10)" + cr
     outer_module_contents = outer_module_contents + "  double_quote = inttochar(34)" + cr + cr
     do ifile = 0, nfiles-1
        unique_name = unique_string("file")
        outer_module_contents = outer_module_contents + cr + ";===========================================================" + cr + cr
        outer_module_contents = outer_module_contents + print_blank_line + cr
        outer_module_contents = outer_module_contents + "  print(" + doublequote + "Now opening the input data file: " + input_filenames(ifile) + doublequote + ")" + cr
        outer_module_contents = outer_module_contents + "  fin_" + unique_name + " = addfile(" + doublequote + input_filepaths(ifile) + input_filenames(ifile) + doublequote + "," + doublequote + "r" + doublequote + ")   ; open output netCDF file" + cr + cr
        outer_module_contents = outer_module_contents + "; retrieve the global attributes from the file" + cr
        outer_module_contents = outer_module_contents + "  fatt_names = getvaratts(fin_" + unique_name + ") " + cr
        outer_module_contents = outer_module_contents + "  if(.not.all(ismissing(fatt_names))) then " + cr + cr
        outer_module_contents = outer_module_contents + print_blank_line + cr
        outer_module_contents = outer_module_contents + "  print(" + doublequote + "The file's global attributes are: " + doublequote + ")" + cr
        outer_module_contents = outer_module_contents + print_blank_line + cr + cr
        outer_module_contents = outer_module_contents + "     do i = 0, dimsizes(fatt_names)-1 " + cr
        outer_module_contents = outer_module_contents + "        print(fatt_names(i) + " + doublequote + " = " + doublequote + " + fin_" + unique_name + "@$fatt_names(i)$) " + cr
        outer_module_contents = outer_module_contents + "     end do " + cr + cr
        outer_module_contents = outer_module_contents + "     delete(fatt_names) " + cr + cr
        outer_module_contents = outer_module_contents + "  end if " + cr + cr
        outer_module_contents = outer_module_contents + "; query the file to see what variables it contains and what the sizes of the file dimensions are " + cr
        outer_module_contents = outer_module_contents + "  varnames = getfilevarnames(fin_" + unique_name + ")  ; Returns an array of file variable names in the file" + cr
        outer_module_contents = outer_module_contents + "  filedimsizes = getfiledimsizes(fin_" + unique_name + ")      ; Returns a list of the sizes of all the dimensions in the file" + cr + cr
        outer_module_contents = outer_module_contents + print_blank_line + cr
        outer_module_contents = outer_module_contents + "  print(" + doublequote + "The file's variables are:" + doublequote + ")" + cr
        outer_module_contents = outer_module_contents + print_blank_line + cr + cr
        outer_module_contents = outer_module_contents + "; query all the named variables in the file " + cr
        outer_module_contents = outer_module_contents + "  vartypes = getfilevartypes(fin_" + unique_name + ",varnames) ; Returns the type of each variable name listed. A missing value is returned for any variable name that doesn't exist in the file" + cr + cr
        outer_module_contents = outer_module_contents + "  nvars = dimsizes(varnames)" + cr + cr
        outer_module_contents = outer_module_contents + "; --------------------------------------------------------------" + cr
        outer_module_contents = outer_module_contents + "; Begin dynamic code generation section for the inner module" + cr + cr
        outer_module_contents = outer_module_contents + "  inner_module_filename = " + doublequote + "dynamic_inner_module_read_vars_" + unique_name + ".ncl" + doublequote + cr
        outer_module_contents = outer_module_contents + "  inner_module_contents = " + doublequote + doublequote + cr + cr
        outer_module_contents = outer_module_contents + "  do ivar = 0, nvars-1" + cr
        outer_module_contents = outer_module_contents + "     inner_module_contents = inner_module_contents + cr + varnames(ivar) + " + doublequote + " = fin_" + unique_name + "->" + doublequote + " + varnames(ivar) + cr " + cr
        outer_module_contents = outer_module_contents + "  end do" + cr + cr
        outer_module_contents = outer_module_contents + "; End dynamic code generation section for the inner module" + cr
        outer_module_contents = outer_module_contents + "; --------------------------------------------------------------" + cr + cr
        outer_module_contents = outer_module_contents + "; Now write the module contents out to a file " + cr
        outer_module_contents = outer_module_contents + "  if(isfilepresent(inner_module_filename)) " + cr
        outer_module_contents = outer_module_contents + "   system(" + doublequote + "/bin/rm -f " + doublequote + " + inner_module_filename" + ")     ; remove a pre-existing file " + cr
        outer_module_contents = outer_module_contents + "  end if  " + cr + cr
        outer_module_contents = outer_module_contents + "  asciiwrite(inner_module_filename,inner_module_contents)" + cr
        outer_module_contents = outer_module_contents + "  delete(inner_module_contents)" + cr + cr
;        outer_module_contents = outer_module_contents + "  system(" + doublequote + "cat " + doublequote + " + inner_module_filename )" + cr
        outer_module_contents = outer_module_contents + "  delete(varnames)" + cr
        outer_module_contents = outer_module_contents + "  delete(nvars)" + cr
        outer_module_contents = outer_module_contents + "  delete(filedimsizes)" + cr
        outer_module_contents = outer_module_contents + "  delete(vartypes)" + cr + cr
;        outer_module_contents = outer_module_contents + "  system(" + doublequote + "cat module_climatology_netcdf_interface.ncl " + doublequote + " + inner_module_filename + " + doublequote + " > executable; mv executable " + doublequote + " + inner_module_filename )" + cr
        outer_module_contents = outer_module_contents + "  loadscript(inner_module_filename)" + cr + cr
     end do     ; end loop over ifile
; End dynamic code generation section
;=====================================================================================================
; Now write the module contents out to a file
     if(isfilepresent(outer_module_filename))
        system("/bin/rm -f " + outer_module_filename)     ; remove a pre-existing file
     end if
     asciiwrite(outer_module_filename,outer_module_contents)
;     system("cat " + outer_module_filename)
     loadscript(outer_module_filename)
  end if
; Now do the rest of your program in the main code block. All the variables which are read in here will reside in memory.
-------------- next part --------------
;********************************************************
; module_netcdf_reader_multiple_prefix_v1.0.ncl
;********************************************************
;
; This module is a modified version of module_climatology_netcdf_reader.ncl,
; which Jonathan Vigh wrote as a graduate student at Colorado State University.
; This version allows the program to automatically read several different model
; files. All of the variables from a particular model are prefixed with a key
; that corresponds to that model. This allows variables of the same name to be
; read from multiple models without confusion. Note that the desired patterns
; that set these keys must be defined by the user below. If the files do not
; contain variables of the same name, then use the regular version.
;
; IMPORTANT DISLCAIMER:
;
; All rights reserved. Absolutely no warranties are implied.
; Use this software at your own risk. I've only tested these
; modules for my own codes and data. I can not promise any
; support for these codes. Feedback is welcome: jvigh@ucar.edu.
;
; This module reads all the variables into memory from
; one or multiple netCDF files whose names have been
; defined in an array before this module is called.
;
; It basically does all the work necessary to bring the
; variables back into memory in the form that they were
; in before they were written out. This code does NOT
; handle logicals, strings, and character arrays with
; non-custom _FillValues (use the RS routines if the
; files contain these types).
;
; In order to use this module to read your files, you must
; do the following:
;
;    1. Load module_climatology_netcdf_interface.ncl in your
;       script after you've loaded gsn_code.ncl, gsn_csm.ncl
;       contributed.ncl, shea_util.ncl, and any other modules
;       containing dependcies for your script. Make sure that
;       you load module_climatology_netcdf_helper_routines.cnl BEFORE
;       you load this module, however. Here's an example of
;       the proper loading sequence:
;
;load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"
;load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"
;load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"
;load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/shea_util.ncl"
;load "./your_other_modules.ncl"
;    2. Define the following three arrays with file information:
;         input_filenames - a list of netcdf files to be read in
;         input_filepaths - a list of the paths to these files
;    3. Load this module before the begin statement of your
;       main program block.
; Now when you the script that loaded this module, all the
; variables contained in your netCDF files will be loaded
; into memory.
;
; Author: Jonathan Vigh, Colorado State University
; Date:   10/16/2009
;
; Modification history:
;    06/17/2012         Made modifications so that this can be used for the climatology project. Changed to use the normal netcdf helper routines (not the RS ones).
;    07/05/2012         Added modifications so that all the variables from a given file are read in under a 'group' whose name is the model keyword. (e.g. GFS, bHWRF).
;
;
;********************************************************
; Note that we do not put the contents of this module within 'begin' and 'end' statements.
; This means that this everything in this module will be global (accessible to all other
; functions and procedures in subsequent modules and the main program which loads this
; module.
  cr          = inttochar(10)
  doublequote = inttochar(34)
  if ( (dimsizes(input_filenames) .eq. 1) .and. (dimsizes(input_filepaths) .eq. 1) ) then
;===================================================================
; Read ONE netCDF file
;===================================================================
     input_filename = input_filenames
     input_filepath = input_filepaths
     print("")
     print("Now opening the input data file: "+input_filename)
     fin = addfile(input_filepath + input_filename,"r")   ; open output netCDF file
; retrieve the global attributes from the file
     fatt_names = getvaratts(fin)
     if(.not.all(ismissing(fatt_names))) then
        print("")
        print("The file's global attributes are: ")
        print("")
        do i = 0,dimsizes(fatt_names)-1
           print(fatt_names(i)+" = "+fin@$fatt_names(i)$)
        end do
     end if
; query the file to see what variables it contains and what the sizes of the file dimensions are
     varnames = getfilevarnames(fin)            ; Returns an array of file variable names in the file
     filedimsizes = getfiledimsizes(fin)        ; Returns a list of the sizes of all the dimensions in the file.
     print("")
     print("The file's variables are:")
     print("")
;     print(varnames)
; query all the named variables in the file
     vartypes = getfilevartypes(fin,varnames)           ; Returns the type of each variable name listed. A missing value is returned for any variable name that doesn't exist in the file.
     nvars = dimsizes(varnames)
;=====================================================================================================
; Begin dynamic code generation section to read in the data from the file
     module_filename = "dynamic_module_read_netcdf.ncl"
     module_contents = ""
     do ivar = 0, nvars-1
        module_contents = module_contents + cr + "  " + varnames(ivar)+" = fin->"+varnames(ivar)
     end do
; Now write the module contents  out to a file
     if(isfilepresent(module_filename))
       system("/bin/rm -f " + module_filename)     ; remove a pre-existing file
     end if
     asciiwrite(module_filename,module_contents)
; End dynamic code generation section
;=====================================================================================================
     loadscript(module_filename)
     system("/bin/rm -f " + module_filename)     ; remove it after is has been used
  else
;===================================================================
; Read MULTIPLE netCDF files
;===================================================================
     nfiles = dimsizes(input_filenames)
;=====================================================================================================
; Begin dynamic code generation section for the outer module
     outer_module_filename = "dynamic_outer_module_read_multiple_netcdf_files.ncl"
     outer_module_contents = ""
     print_blank_line = "  print(" + doublequote + doublequote + ")"
     outer_module_contents = outer_module_contents + "  cr           = inttochar(10)" + cr
     outer_module_contents = outer_module_contents + "  double_quote = inttochar(34)" + cr + cr
     outer_module_contents = outer_module_contents + "  underscore   = inttochar(95)" + cr + cr
     do ifile = 0, nfiles-1
;        unique_name = unique_string("file")
        if (isStrSubset(input_filenames(ifile),"gfs")) then
           unique_name = "GFS"
        end if
        if (isStrSubset(input_filenames(ifile),"bhwrf")) then
           unique_name = "BHWRF"
        end if
        if (isStrSubset(input_filenames(ifile),"ecmwf")) then
           unique_name = "ECMWF"
        end if
        outer_module_contents = outer_module_contents + cr + ";===========================================================" + cr + cr
        outer_module_contents = outer_module_contents + print_blank_line + cr
        outer_module_contents = outer_module_contents + "  print(" + doublequote + "Now opening the input data file: " + input_filenames(ifile) + doublequote + ")" + cr
        outer_module_contents = outer_module_contents + "  fin_" + unique_name + " = addfile(" + doublequote + input_filepaths(ifile) + input_filenames(ifile) + doublequote + "," + doublequote + "r" + doublequote + ")   ; open output netCDF file" + cr + cr
        outer_module_contents = outer_module_contents + "; retrieve the global attributes from the file" + cr
        outer_module_contents = outer_module_contents + "  fatt_names = getvaratts(fin_" + unique_name + ") " + cr
        outer_module_contents = outer_module_contents + "  if(.not.all(ismissing(fatt_names))) then " + cr + cr
        outer_module_contents = outer_module_contents + print_blank_line + cr
        outer_module_contents = outer_module_contents + "  print(" + doublequote + "The file's global attributes are: " + doublequote + ")" + cr
        outer_module_contents = outer_module_contents + print_blank_line + cr + cr
        outer_module_contents = outer_module_contents + "     do i = 0, dimsizes(fatt_names)-1 " + cr
        outer_module_contents = outer_module_contents + "        print(fatt_names(i) + " + doublequote + " = " + doublequote + " + fin_" + unique_name + "@$fatt_names(i)$) " + cr
        outer_module_contents = outer_module_contents + "     end do " + cr + cr
        outer_module_contents = outer_module_contents + "     delete(fatt_names) " + cr + cr
        outer_module_contents = outer_module_contents + "  end if " + cr + cr
        outer_module_contents = outer_module_contents + "; query the file to see what variables it contains and what the sizes of the file dimensions are " + cr
        outer_module_contents = outer_module_contents + "  varnames = getfilevarnames(fin_" + unique_name + ")  ; Returns an array of file variable names in the file" + cr
        outer_module_contents = outer_module_contents + "  filedimsizes = getfiledimsizes(fin_" + unique_name + ")      ; Returns a list of the sizes of all the dimensions in the file" + cr + cr
;        outer_module_contents = outer_module_contents + print_blank_line + cr
;        outer_module_contents = outer_module_contents + "  print(" + doublequote + "The file's variables are:" + doublequote + ")" + cr
;        outer_module_contents = outer_module_contents + "  print(varnames)" + cr
        outer_module_contents = outer_module_contents + print_blank_line + cr + cr
        outer_module_contents = outer_module_contents + "; query all the named variables in the file " + cr
        outer_module_contents = outer_module_contents + "  vartypes = getfilevartypes(fin_" + unique_name + ",varnames) ; Returns the type of each variable name listed. A missing value is returned for any variable name that doesn't exist in the file" + cr + cr
        outer_module_contents = outer_module_contents + "  nvars = dimsizes(varnames)" + cr + cr
        outer_module_contents = outer_module_contents + "; --------------------------------------------------------------" + cr
        outer_module_contents = outer_module_contents + "; Begin dynamic code generation section for the inner module" + cr + cr
        outer_module_contents = outer_module_contents + "  inner_module_filename = " + doublequote + "dynamic_inner_module_read_vars_" + unique_name + ".ncl" + doublequote + cr
        outer_module_contents = outer_module_contents + "  inner_module_contents = " + doublequote + doublequote + cr + cr
        outer_module_contents = outer_module_contents + "  do ivar = 0, nvars-1" + cr
        outer_module_contents = outer_module_contents + "     inner_module_contents = inner_module_contents + cr + " + doublequote + unique_name + doublequote + " + underscore + varnames(ivar) + " + doublequote + " = fin_" + unique_name + "->" + doublequote + " + varnames(ivar) + cr " + cr
        outer_module_contents = outer_module_contents + "  end do" + cr + cr
        outer_module_contents = outer_module_contents + "; End dynamic code generation section for the inner module" + cr
        outer_module_contents = outer_module_contents + "; --------------------------------------------------------------" + cr + cr
        outer_module_contents = outer_module_contents + "; Now write the module contents out to a file " + cr
        outer_module_contents = outer_module_contents + "  if(isfilepresent(inner_module_filename)) " + cr
        outer_module_contents = outer_module_contents + "   system(" + doublequote + "/bin/rm -f " + doublequote + " + inner_module_filename" + ")     ; remove a pre-existing file " + cr
        outer_module_contents = outer_module_contents + "  end if  " + cr + cr
        outer_module_contents = outer_module_contents + "  asciiwrite(inner_module_filename,inner_module_contents)" + cr
        outer_module_contents = outer_module_contents + "  delete(inner_module_contents)" + cr + cr
;        outer_module_contents = outer_module_contents + "  system(" + doublequote + "cat " + doublequote + " + inner_module_filename )" + cr
        outer_module_contents = outer_module_contents + "  delete(varnames)" + cr
        outer_module_contents = outer_module_contents + "  delete(nvars)" + cr
        outer_module_contents = outer_module_contents + "  delete(filedimsizes)" + cr
        outer_module_contents = outer_module_contents + "  delete(vartypes)" + cr + cr
;        outer_module_contents = outer_module_contents + "  system(" + doublequote + "cat module_climatology_netcdf_interface.ncl " + doublequote + " + inner_module_filename + " + doublequote + " > executable; mv executable " + doublequote + " + inner_module_filename )" + cr
        outer_module_contents = outer_module_contents + "  loadscript(inner_module_filename)" + cr + cr
     end do     ; end loop over ifile
;     outer_module_contents = outer_module_contents + "  list_vars()" + cr + cr
; End dynamic code generation section
;=====================================================================================================
; Now write the module contents out to a file
     if(isfilepresent(outer_module_filename))
        system("/bin/rm -f " + outer_module_filename)     ; remove a pre-existing file
     end if
     asciiwrite(outer_module_filename,outer_module_contents)
;     system("cat " + outer_module_filename)
     loadscript(outer_module_filename)
  end if
; Now do the rest of your program in the main code block. All the variables which are read in here will reside in memory.
------------------------------
Message: 5
Date: Sat, 21 Jul 2012 11:26:43 +0800
From: " ?? " <wanghai@ouc.edu.cn>
Subject: errors in using 'addfiles'
To: ncl-talk@ucar.edu
Message-ID: <BECCKPLWURHSUODZVLQLMGCRCDUL.wanghai@ouc.edu.cn>
Content-Type: text/plain; charset="gb2312"
Hi all,
I was trying to use NCL to read the cmip5 data with "addfiles" command. But I  encountered some errors that really confused me.
Here is my scripts:
>fils = systemfunc("ls ./zg/zg_Amon_GFDL-HIRAM-C180_amip_r1i1p1_*.nc")
>f    = addfiles(fils,"r")
>ListSetType (f,"cat")
>HGT  = f[:]->zg          (this is line 8)
>printVarSummary(HGT)
and here is the error information:
fatal: Aggregating variable zg from file list variable f as specified would exceed maximum NCL variable size
fatal: Execute: Error occured at or near line 8 in file hgt.ncl
By the way, the version of my NCL is v6.0.0 and I think this version of NCL could support the data that larger than 2GB (my data is 4.7GB)
I am expecting for you advise. Thank you very much
Best regards,
                                                            Hai Wang
----------------
Hai Wang
College of Physical and Environmental Oceanography,
Ocean University of China
238, Songling Road, Qingdao, Shandong, China, 266100
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20120721/4872ad23/attachment.html
------------------------------
_______________________________________________
ncl-talk mailing list
ncl-talk@ucar.edu
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
End of ncl-talk Digest, Vol 104, Issue 42
*****************************************
_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Sat Jul 21 16:12:16 2012

This archive was generated by hypermail 2.1.8 : Tue Jul 24 2012 - 09:48:26 MDT