NCL Home > Documentation > Functions > System tools

subprocess

Executes a shell command as a concurrent subprocess to NCL.

Available in version 6.5.0 and later.

Prototype

	function subprocess (
		command [1] : string   
	)

	return_val [1] :  integer

Arguments

command

A singly dimensioned string containing a shell command to execute.

Return value

Returns a numeric ID associated with the subprocess. This ID may be used with the subprocess_wait function to determine whether the subprocess has finished or to wait for it to finish.

Description

This procedure passes the string command to the shell environment for concurrent execution in a new subprocess. Control is returned immediately to NCL and the calling script continues its execution. Any output from the command to stdout/stderr will be intermingled with output from the calling script unless the shell command employs I/O redirection (see Examples below).

Using the subprocess command, an NCL script can perform several tasks in parallel. Typical use-cases might be the parallel execution of multiple independent NCL scripts, or executing multiple instances of a plotting script to generate plot-frames intended for an animation.

The default shell used to execute command is /bin/sh (Bourne Shell). A different shell may be specified in command; to use a different shell, such as the C-Shell, commands are enclosed by single quotes (') to prevent the Bourne Shell from interpreting the commands. If the C-Shell command itself contains single quotes, they must be escaped with a '\'. See Example 5 for the system command.

Examples

Example 1

Creating an animation. This example consist of two scripts. One is a simple "worker" script that takes a timestep as a parameter on the command-line and generates a contour plot of a variable for that time. It gets invoked by an outer "driver" script that coordinates running multiple instances of the worker script and then uses Imagemagick to generate an animation from the individual frames. The driver employs a simple gating mechanism to run some number of plot instances concurrently to keep the machine busy, but not so many as to overwhelm the system. Informal experiments with this techniquehave suggested that the number of concurrent plots could be on the order of the number of processor cores the machine has.

Driver:

begin
  MAX_CONCURRENT = 8
  numTimeSteps = -1
  ;
  ; determine how many time steps from the file...
  ;
  f = addfile("sst8292.nc", "r")
  dimNames = getvardims(f)
  dimSizes = getfiledimsizes(f)
  do i=0, dimsizes(dimNames)-1
    if (dimNames(i).eq."time") then
      numTimeSteps = dimSizes(i)
      break
    end if
  end do
  print("Number of timesteps: " + numTimeSteps)
  delete(f)             ; no longer needed
  ;
  ; begin generating plot frames...
  ;
  numPlotsCompleted = 0
  numJobsActive = 0
  do i=0,numTimeSteps-1

    ; don't allow more than MAX_CONCURRENT processes at a time....
    do while (numJobsActive.ge.MAX_CONCURRENT)
      ; check if any tasks have completed; block until something completes...
      pid = subprocess_wait(0, True)
      if (pid.gt.0) then
        numPlotsCompleted = numPlotsCompleted + 1
        numJobsActive = numJobsActive - 1
        break
      end if
    end do

    print("working on frame: " + i)
    command = "ncl contourSST.ncl " + str_get_sq() + "timestep=" + i + str_get_sq() + " >/dev/null"
    pid = subprocess(command)
    numJobsActive = numJobsActive + 1
  end do
  ;
  ; need to wait for all pending jobs to complete...
  ;
  do while (numPlotsCompleted.lt.numTimeSteps-1)
      pid = subprocess_wait(0, True)
      if (pid.gt.0) then
        numPlotsCompleted = numPlotsCompleted + 1
      end if
  end do
  ;
  ; use Imagemagick to create the animation from the *png frames...
  ; 
  system("convert -delay 100 -loop 0 sst*.png sst.gif")

end

Plotting script: