NCL > What's New

What's new in the current release

[previous releases | next release]

Version 6.0.0 - May 30, 2011 - [download]

See what's new since 6.0.0-beta

Variables greater than or equal to 2 gigabytes (GB) in size are allowed

Prior to version 6.x, NCL variables could not be greater than or equal to 2 GB in size. This restriction has been removed on 64-bit systems (the restriction remains for 32-bit systems).

Also with this upgrade, dimension sizes will be of type "long" on 64-bit systems if any dimension size is >= 2 GB.

Important note about *.so files generated by WRAPIT

In this release, you will need to regenerate any *.so files generated by older versions of WRAPIT, as these old files will not be compatible with the new NCL software.

If you try to use the old files with the new NCL, you will get a fatal message that looks like this: 

fatal:Number of elements of dimension (0) of argument (0) is (1) in 
      function (stuff), expected (4582672464) elements

Change to default missing values

The default missing values for all numeric variable types has been changed. For full information on this, see the "data types" section in the NCL Reference Manual.

This is in line with the default missing values for NetCDF, and some of the old default values were too small to be safely out of the range of potential calculations.

Here's a table of the old and new missing values:

Variable type Old missing value New missing value
byte* 0xff -127
ubyte -- 255
short -99 -32767
ushort 0 65535
integer -999 -2147483647
uint 0 4294967295
long -9999 -2147483647
ulong 0 4294967295
int64 -99999999 -9223372036854775806
uint64 0 18446744073709551614
float -999 9.96921e+36
double -9999 9.969209968386869e+36
character* 0 0x00

* The missing values for bytes and characters are rather different. This is because an NCL byte is now a signed byte (it was unsigned before), and an NCL character is now an unsigned byte (it was signed before). This change keeps us in line with the definition of a NetCDF byte and character.

Possible "gotchas" in default missing value change

The default missing value change could cause some backwards incompatibility in your NCL scripts. Here are some "gotchas" to look for and correct:

  • Do not use hard-coded values to check if your data contains any missing values.

    For example, if you have a float variable "x" that was assigned a default missing value of -999 via some other calculation, *and* you have code like this: 

        if(x(i).eq.-999)) then
      ...do something...

    then your code may no longer work in V6.0.0. You should instead use functions like ismissing to check for missing values: 

        if(ismissing(x(i))) then
      ...do something...

  • Be careful with checking for missing data by assuming it is less than zero.

    If you have data that is supposed to always be positive, and you check for missing values by assuming the default value is less than zero, then this may no longer work in V6.0.0. For example, the default missing values for floats and doubles were negative in V5.x, and are now positive in V6.x.

  • Be careful with sorting data that may contain default missing values.

    Functions like qsort and dim_pqsort sort missing values like regular values. In V6.x, the default float and double missing values are now positive, so they will be sorted differently than in V5.x where they were negative.

  • Set the "_FillValue" attribute if you are reading ASCII data.

    If you are reading data from an ASCII file that potentially contains missing values, and if these missing values are equal to the *old* default values, then you may need to set the _FillValue attribute explicitly after reading the data in.

    Functions like asciiread and readAsciiTable automatically assign a _FillValue attribute based on the type of the variable you're reading in. If you are assuming that the default missing value is -999 for floats, say, then your code will not work properly.

    For example, if you have an ASCII file with float values, and you have "-999" in the file to represent missing values, then you will need to explicitly set the _FillValue attribute to -999 after you read it in, because now the default missing value for a float will be "9.96921e+36": 

       data = asciiread("file.txt",-1,"float")
   data@_FillValue = -999

We have provided two ways to get at the old default missing values if you are not ready to change your code. The first way is via a procedure call: 

  set_default_fillvalue("all","ncl_v5")

The second way is via a command line option when you run NCL.

Change to byte and character types

In NCL versions 5.x and earlier, an NCL byte was unsigned, and an NCL character was signed. To make things more consistent, and to stay in line with bytes and characters in NetCDF, these two types have switched meaning: an NCL byte is now signed and an NCL character is unsigned.

We've introduced a new type---"ubyte" (unsigned byte)---if you need the functionality of the old-style NCL byte.

For more information about NCL data types, see the "data types" section in the NCL Reference Manual.

New functions

  • cd_calendar (V6.0.0) - converts a mixed Julian/Gregorian date to a UT-referenced date. This is a replacement for ut_calendar

  • cd_inv_calendar (V6.0.0) - converts a UT-referenced date to a mixed Julian/Gregorian date. This is a replacement for ut_inv_calendar

  • gsn_csm_blank_plot (V6.0.0-beta) - creates a "blank" plot with tickmarks pointing outward.

  • isubyte (V6.0.0-beta) - returns True if the input is an unsigned byte.

  • isunsigned (V6.0.0-beta) - returns True if the input is any of the unsigned types (ubyte, ushort, uint, ulong, uint64).

  • default_fillvalue (V6.0.0-beta) - returns the default missing value for the given variable type.

  • namedcolor2rgb (V6.0.0-beta) - Returns the RGB triplets of the given list of named colors.

  • remap_elements (V6.0.0) - Remaps values on one grid to a rectilinear latitude-longitude grid.

  • set_default_fillvalue (V6.0.0-beta) - allows you to change the default missing value for a particular variable type, or change all the default missing values to the same values that were used in NCL version 5.x.

  • show_ascii (V6.0.0) - prints the ASCII table to the screen.

  • span_named_colors (V6.0.0-beta) - Creates an RGB array that is a span between given list of named colors.

  • str_match_ic (V6.0.0) - returns a list of strings that contain the given substring (case insensitive).

  • str_match_ind (V6.0.0) - returns the indexes into a list of strings that contain the given substring (case sensitive).

  • str_match_ind_ic (V6.0.0) - returns the indexes into a list of strings that contain the given substring (case insensitive).

  • str_split_by_length (V6.0.0-beta) - splits strings given a scalar length or array of lengths.

  • toubyte (V6.0.0-beta) - converts input to an unsigned byte.

Updated functions

  • Numerous functions have been upgraded to allow dimension sizes and index values to be integers or longs. On a 64-bit system, dimension sizes of type long allows you to have variables >= 2 GB.

    Here is a list of functions that have been upgraded to allow for long input and/or output. Not all updated functions are listed here; just the most widely-used ones:

    • conform_dims (V6.0.0-beta) - the first argument can be a long.

    • dim_gbits, getbitsone (V6.0.0-beta) - now accepts int8, uint8, and ushort input.

    • dimsizes (V6.0.0-beta) - longs will be returned if you're on a 64-bit machine and the product of your dimension sizes is >= 2 GB.

    • fspan (V6.0.0-beta) - the third argument can be a long.

    • ind, minind, maxind (V6.0.0-beta) - these functions will return longs if you're on a 64-bit system and any of the index values are >= 2 GB.

    • ind_resolve (V6.0.0-beta) - longs will be returned if any of the index values are >= 2 GB. Also, the input can now be integer or long.

    • ispan (V6.0.0-beta) - any of the input can be of type int, long, or int64. Also, the return type can be int, long, or int64, depending on the input types.

    • new (V6.0.0-beta) - the input dimension sizes can be longs.

    • num (V6.0.0-beta) - a long will be returned if you're on a 64-bit system and the count is >= 2 GB.

    • onedtond (V6.0.0-beta) - the second argument can be longs.

    • sizeof (V6.0.0-beta) - the size returned will be a long if you're on a 64-bit machine and the size is >= 2 GB.

  • rcm2points / rcm2points_Wrap (V6.0.0) - the simple search algorithm used by these functions is not capable of handling all cases. The result is that, sometimes, there are non-interpolated points in the initial interpolation pass. Beginning with V6.0.0: any interior points not interpolated in the initial interpolation pass will be filled by searching all points and using an inverse distance squared algorithm.

  • rcm2rgrid / rcm2rgrid_Wrap (V6.0.0) - the simple search algorithm used by these functions is not capable of handling all cases. The result is that, sometimes, there are small gaps in the interpolated grids. Beginning with V6.0.0: any interior points not interpolated in the initial interpolation pass will be filled using linear interpolation. In some cases, edge points may not be filled.

  • wrf_interp_3d_z, wrf_user_intrp3d (V6.0.0-beta) - the missing value was changed from -999999 to the new default missing value for a float/double (9.96921e+36).

  • wrf_user_list_times, wrf_user_getvar, wrf_user_ij_to_ll, wrf_user_ll_to_ij (V6.0.0) - these functions have been updated to allow the first argument to be either a file opened with addfile, or a list of files opened with addfiles.

Deleting multiple variables

In alpha test mode: you can now use delete to delete multiple variables of mixed types and sizes. Use the special syntax "[/" and "/]" around the variables you want to delete: 

  lat  = fspan(-90,90,64)
  lon  = fspan(-180,180,128)
  data = random_uniform(-10,10,(/10,20,30/))
  varnames = (/"T","P","U","V"/)    ; 1D string array
  . . .
  delete([/data,varnames,lat,lon/])

  print(lon(0))    ; This will produce an error that "lon" is undefined

Keywords added

  • enumeric
  • group
  • int64
  • snumeric
  • ubyte
  • uint
  • uint64
  • ulong
  • ushort
  • Missing

New color tables

Emilie Vanvyve of NCAR/RAL contributed eight new color tables. You can download these tables now and use them. Click on the "download now" link of the desired table, and then go to the "Move file to appropriate directory" section of the "create your own color table" page for information on adding this new table to your current version of NCL.

BlueWhiteOrangeRed
(download now) 		BlueYellowRed
(download now) 		GrayWhiteGray
(download now) 		OceanLakeLandSnow
(download now)
WhiteBlue
(download now) 		WhiteBlueGreenYellowRed
(download now) 		WhiteGreen
(download now) 		WhiteYellowOrangeRed
(download now)

There are now 101 color tables available.

New graphical resources

cnGridBoundFillColor
This resource sets the color to use for filling areas outside the boundaries of the data grid, but within the viewport and the projectable areas of the underlying map, if any.
cnGridBoundFillPattern
This resource sets the fill pattern index used to fill areas outside the boundaries of the data grid, but within the viewport and the projectable areas of the underlying map, if any.
cnGridBoundFillScaleF
This resource controls the scaling of the pattern used for areas outside the boundaries of the data grid, but within the viewport and the projectable areas of the underlying map, if any.
cnOutOfRangeFillColor
This resource sets the color to use for filling areas within the viewport but outside the projectable area of the underlying map, if any.
cnOutOfRangeFillPattern
This resource sets the fill pattern index used to fill areas within the viewport but outside the projectable area of the underlying map, if any.
cnOutOfRangeFillScaleF
This resource controls the scaling of the pattern used for areas within the viewport but outside the projectable area of the underlying map, if any.

Also, the default values of cnMissingValFillColor and cnMissingValFillPattern have changed: cnMissingValFillColor formerly defaulted to Background and cnMissingValFillPattern defaulted to HollowFill. Now the defaults have respectively changed to Transparent and SolidFill. Since the effect is the same in either case, this change is not anticipated to affect very many existing scripts. The change was made to simplify the interactions between these resources and the above newly added resources. One advantage of this change is that, assuming contour fill is enabled, you can get visible fill of the missing value areas by modifying only one resource instead of two.

Change to naming of PNG files

The PNG file format does not support multiple images per file. In NCL 5.2.x and earlier, multiple plots written to a PNG workstation resulted correspondingly in multiple files, named with a 6-digit sequence number appended to the user-specified filename (file.000001.png, file.000002.png, etc). In cases where only one plot is generated, this behavior was unnecessary and deemed undesirable by a number of users.

With NCL 6.0.0, no sequence number is appended when only a single plot is generated (file.png). As before, when multiple plots are written, sequence numbers are appended to all filenames, including the first for consistency.

Bugs fixed

  • (In version 6.0.0-beta only) If you use a "/" in conjunction with reading a variable off a file: 

      a   = addfile("atmos.nc","r")
  lev = a->lev/100

    You will get an unexpected syntax error: 

    fatal:Either file (a) isn't defined or variable (lev/100) is not a
variable in the file

    This is an error due to the implementation of reading "groups" in HDF5, which can be accessed with a "/". NCL thinks you are trying to access a group called "lev/100", rather than trying to divide the "lev" values by 100.

  • In the process of testing NCL 6.0.0, we ran a number of memory debugging codes (valgrind, purify) and fixed many memory leaks.

  • Fixed a bug in which the NCL polygon code was not being handled correctly on spherical surfaces, potentially causing the fill to "bleed".

  • Fixed a bug with the "record"/"stop" commands that caused them to abort and produce a bunch of errors on a Linux system.

  • Fixed a bug with ncl_filedump not working on 2D character arrays or improperly deleting a local variable if a string variable was being processd.

  • Fixed some GRIB bugs:

    • GRIB user-defined parameter table wasn't read correctly when there was an entry for parameter 0.
    • GRIB grid 242 didn't have the correct values for Dx and Dy.

  • Fixed a bug with drawing triangular mesh contours to the full viewport.

  • Fixed a bus error caused by initializing an NCL variable with an invalid value.

  • Fixed a bug with logical values not being converted to strings correctly if set to the default missing value.

  • Fixed a bug in which if NCL_DEF_SCRIPTS_DIR environment variable was set, NCL scripts were not loaded in a predictable order. Scripts are now loaded in ascii case-sensitive order (numbers before letters, upper case before lower case). Files that do not end with '.ncl' are simply ignored.

  • asciiread - fixed inconsistent behavior where this function was setting a missing value if the dimensions were explictly set, and not if the dimensions were specified with "-1". Note that now that the default missing values have been changed, you will see the new default missing values returned for this function.

  • genNormalDist - this function was calculating PI incorrectly.

  • getfiledimsizes - this function was incorrectly returning longs if the individual dimension sizes were less than 2 GB.

  • gsn_csm_pres_hgt_vector - the vector array needs to have pressure values in "mb", so added a check for this.

  • obj_anal_ic - this routine had a bug if the input arrays had different missing values.

  • rcm2points_Wrap / rcm2points_Wrap - details soon

  • rcm2rgrid_Wrap - fixed a serious bug in which the lat/lon input and output arrays were swapped. rcm2rgrid did not have this bug.

  • str_match - this function was case insensitive and it should have been case sensitive. Use the new str_match_ic if you need case-insensitivity matching.

  • str_split - this function was freeing non-allocated memory.
  • ut_calendar / ut_inv_calendar - there are a number of bugs in these functions, due to the underlying code in UDUNITS2 not being supported anymore. Use the new functions cd_calendar / cd_inv_calendar instead.

What's new/updated since 6.0.0-beta

New functions:

Updated functions:

New resources:

  • cnGridBoundFillColor
  • cnGridBoundFillPattern
  • cnGridBoundFillScaleF
  • cnOutOfRangeFillColor
  • cnOutOfRangeFillPattern
  • cnOutOfRangeFillScaleF

Improved functionality:

  • The order in which scripts contained in the NCL_DEF_SCRIPTS_DIR directory are loaded is no longer undefined. Now, based on their names, the scripts are loaded in ascii case-sensitive order (numbers before letters, upper case before lower case). Files in the directory that do not have a ".ncl" suffix are now simply ignored, rather than causing a fatal error. A warning is generated if there are no valid scripts in the directory, and a script that fails to load properly causes a fatal error.