Cool tips and not-so-cool quirks of NCL

• Tip: quickly converting a number to a string
• Tip: using LaTeX fonts
• Tip: creating a new variable without a _FillValue attribute
• Tip: quickest way to convert GRIB1, GRIB2, HDF 4, HDF-EOS 2 file to netCDF file
• Tip: converting a large file to a netCDF file
• Tip: creating a double variable without using "new"
• Tip: writing a scalar to a netCDF file
• Tip: creating unique random subscripts
• Tip: including the double quote (") character in a string
• Quirk: behavior of lat/lon limits in map
• Quirk: behavior of negation operator
• Quirk: memory issues with creating lots of strings
• Quirk: error: line -1 before or near...
• Quirk: opening lots of files
• Quirk: workspace reallocation error message
• Quirk: ARSCAM/ARPRAM ALGORITHM FAILURE

• Quickly converting a number to a string

    x   = 3.14
    str = "" + x
  

  Use sprinti or sprintf for formatted strings.

  
  

• Using LaTeX fonts

  Let's say that you need to include an encapsuled postscript generated by NCL in your LaTeX document. If this image contains character strings, you may want them to be written in your own LaTeX style. Here is a solution contributed by Raphael Pierre, a PhD student at Fresnel Institute in France:

  As an example, let's consider the x-axis label of some plot.

  1. Adapting the NCL source file.

     A priori, no matter the font you specify in your .ncl file, just pick one.

            res@tiXAxisFont = 21 ; Helvetica.
     

     Choose a dummy character (you'll have to reuse it afterwards) but avoid character strings.

            res@tiXAxisString = "a"
     

     The important step is the one that follows.

            res@tiXAxisFontQuality = "Low"
     

     This option forces NCL to include the font in the output file, the .eps image.

     To ensure this has been done correctly, open the .eps file with a text editor and check if a line like that:

         %%DocumentFonts: Helvetica
     

     appears at the bottom.

  2. Adapting the LaTeX source file.

     Add the usual packages for including graphics (e.g. graphicx). Add the PSfrag package (available on CTAN and in usual LaTeX distros) which will permit you to replace the dummy character by the adequate string.

        \usepackage{psfrag}
     

     Use the appropriate command.

            \begin{figure}
            \psfrag{a}{$x$}
            \includegraphics{figure.eps}
            \end{figure}
     

     And you're done.

  3. Creating a new variable without a _FillValue attribute

       x = new(5, float, "No_FillValue")
     

     
     

  4. Quickest way to convert GRIB 1, GRIB 2, HDF 4, HDF-EOS 2 file to netCDF file

     On UNIX command line:

       ncl_convert2nc your_file.xxx
     

     where xxx is file suffix: "grb", "hdf", or "hdfeos".

     
     

  5. Converting a large file to a netCDF file

     If you are using ncl_convert2nc to convert a supported file (like GRIB) to a netCDF file, and getting the error message:

        ncendef: ncid 0: NetCDF: One or more variable sizes violate format
        constraints
     

     then you may need to include the "-L" option:

         ncl_convert2nc U12345.grb -L
     

  6. Creating a double variable without using "new"

       x = 1.d      ; x = 1d   (also acceptable)
     

     
     

  7. Writing a scalar to a netCDF file

     First way, using "ncl_scalar" as a dimension name:

     spherical = new(1,"character") 
     spherical!0 = "ncl_scalar" 
     spherical = integertocharacter(1)
     ncid->spherical = spherical 
     

     Second way, using filevardef:

     filevardef(ncid, "spherical","character","ncl_scalar") 
     ncld->spherical = integertocharacter(1) 
     

     Note that for either of these methods, if you look at this file in NCL it will appear as if there is a dimension named "ncl_scalar". In an ncdump of the file, however, the dimension will not appear, so the variable is a true NetCDF scalar.

  8. Creating unique random subscripts

     Say it is desired to generate 1000 unique subscripts.

       r = random_uniform(0,1,1000)  ; random numbers
       i = dim_pqsort(r, 1)          ; return indices
     

     
     

  9. Including the double quote (") character in a string

     Since NCL uses double quotes to denote strings, you have to do an extra step to get quotes into a string:

       quote = inttochar(34)
       print("This is a " + quote + "string" + quote + " with double quotes around it.")
       print("'Single' quotes are no big deal.")
     

• Behavior of lat/lon limits in map

  Whenever you plot a subset of longitudes, and you are getting seemingly strange results, you likely need to set the resource mpCenterLonF, which will tell ncl where you want the center longitude of the plot to be. Usually, this should be set to the halfway point between mpMinLonF and mpMaxLonF:

    res@mpCenterLonF = (res@mpMinLonF+res@mpMaxLonF)/2.
  

• Behavior of negation operator

  The negation operator ('-') has the highest precedence in an NCL expression, potentially producing unexpected results. For example, the following expression:

      x = - 3^2
  

  yields a value of 9 because it behaves as:

      x = (-3)^2
  

  Note how the use of parentheses can change the results:

      print(- (3+2)^2)      ----> yields 25
  

  whereas:

      print(- ((3+2)^2))    ---->  yields -25
  

  Finally, if the '-' symbol appears between two expressions or numbers, then it is treated as a "minus" rather than negation. Hence the following yields a value of 9:

      x = 0 - 3^2      ; equivalent to  x = (0) - (3^2)
  

  
  

• Memory issues with creating lots of strings

  If you create strings in NCL, the memory for them never gets freed internally, even if you delete them. If you create lots of strings (and we do mean a lot), then, you may run into memory problems or notice an increased slow down as the script progresses.

  NCL permanently (by design) stores every unique string it encounters in a hash table and assigns it a unique integer id, which is passed around instead of a pointer reference to the string itself. This simplifies programming for tasks such as string comparison. In the usual case for NCL, this has worked pretty well.

  Unfortunately, there is a drawback to this approach. Let's say you have a do loop that is executed many times (like 2 million). Further assume that inside this loop, you build up strings from small components and that each of these strings is unique.

  As you go through this big loop, we have found that the virtual memory required to hold these unique strings tends to grow, and as the hash table gets larger, it takes more and more processing time to distinguish each string as unique or not. Using performance monitoring tools we found that in one case, when a string-intensive program first started, it was spending about 25% of its time locating strings in the hash table. That had grown to 85% of the time towards the end of the loop.

  We have been doing some research into fixing this quirk, but meanwhile, the best we can do is to suggest trying to find ways around having to create a lot of unique strings. Feel free to email ncl-talk@ucar.edu if you have any concerns or questions about this.

  
  

• error: line -1 before or near...

  If you get this error message when running an NCL script:

    fatal:syntax error: line -1 before or near
   
    ^
   
    fatal:error in statement
  

  then you probably have a "begin" statement in your script, but no "end" statement.

  
  

• Opening lots of files

  If you need to open lots of files with either addfile or addfiles, then you should probably consider calling setfileoption and setting "SuppressClose" to False.

  The number of files you can have open at any time is dependent on your system. A general limit seems to be around 1024.

• Workspace reallocation error message

  If you get an error message that looks something like this:

      fatal:ContourPlotDraw: Workspace reallocation would exceed maximum size
      16777216
      fatal:ContourPlotDraw: draw error
      fatal:ContourPlotDraw: draw error
      fatal:PlotManagerDraw: error in plot draw
      fatal:_NhlPlotManagerDraw: Draw error
  

  then you need to bump up your workspace size. First, retrieve the default "workspace id", and then set the "wsMaximumSize" resource to a value larger than 16777216. To do this, add the following code right after you create a workstation:

      setvalues NhlGetWorkspaceObjectId() 
        "wsMaximumSize" : 50000000
      end setvalues 
  

  You can permanently bump up the workspace size by setting this resource in your ~/.hluresfile:

  *wsMaximumSize : 50000000 
  

  You may also need to play with the 50000000 value. This is just a first guess.

  If you are drawing a contour plot, you might instead try switching to raster contouring, by setting the resource "cnFillMode" to "RasterFill".

  
  

• ARSCAM/ARPRAM ALGORITHM FAILURE

  If you get an error message that says "ARSCAM/ARPRAM ALGORITHM FAILURE", this usually means something happened in a very low algorithm for generating the contours, caused by a precision error. There's a more detailed description available.

  Unfortunately, there's no instant fix for this. We found that if you tweak certain things, like the size of the plot, the contour levels, the number of contours, etc, you might be able to get rid of this error message.

  Meanwhile, we are interested in hearing from folks who run into this problem. You can help us debug the problem by setting an additional resource along with your other contour resources:

    res@cnDumpAreaMap = True
  

  When you run your script, it should echo a message like:

     Dumping areamap to file NhlAreaMap.0
  

  Please send the "NhlAreaMap.0" file to Mary Haley and we'll take a look. We may also contact you to ask for more information.