Re: Parallel NCL

From: <Oliver.Fuhrer_at_nyahnyahspammersnyahnyah>
Date: Mon Jul 18 2011 - 00:37:00 MDT

Dear ParVis team,

A lot of our very time intensive scripts are of the form...

read constant data
do i = 1, nplot
  read data
  process data
  make plot
end do

...where the loop is over a number of timesteps or a number of different plots. Accelerating the processing step by parallelization surely will give these scripts a boost. But it usually also doesn't exploit the inherent parallelism that is very often present, namely that the work inside the loop body is independent and could be done in parallel. So having something like...

read constant data
parallel do i = 1, nplot
  read data
  process data
  make plot / write data
end parallel do

...would be a much better way to parallelize. As several people have pointed out, this strategy is already implemented "by hand" by several people who run time critical stuff by splitting the script into several scripts and ncl instances.

I see that this might be very tricky to implement depending on the internal runtime execution model of NCL. Also, one would probably need to specify scoping of variables. For example all variables outside of the loop body are "shared" and all variables inside the loop body (including the loop index) are "private".

Kind regards,
Oli

> -----Original Message-----
> From: ncl-talk-bounces@ucar.edu
> [mailto:ncl-talk-bounces@ucar.edu] On Behalf Of Mary Haley
> Sent: Dienstag, 12. Juli 2011 23:18
> To: ncl-talk@ucar.edu forum
> Subject: Parallel NCL
>
> Dear NCL Users,
>
> A new joint project (ParVis) between NCAR and DOE labs is
> working on a parallel version of NCL! We will be
> implementing parallel versions of NCL built-in functions.
>
> To help guide our effort, we'd like to hear which functions
> you have found either are too slow or take too much memory
> for the data sets you are applying them to. Please also let
> us know how big the data is and what kind of machine you're
> trying to run on (OS, processor, memory size) for those functions.
>
> Another goal is to support mathematical operations on native
> grids so you do not have to convert to lat-lon. Please let
> us know which functions/capabilities you would like to see
> supported first on native grids.
>
> Finally, please let us know about any new functions you would
> like to see implemented. What is your "wish list"?
>
> Send replies to ncl-talk.
>
> For more information, see http://trac.mcs.anl.gov/projects/parvis
>
> Regards,
> ParVis team.
>

 
________________________________________

Oliver Fuhrer
Numerical Models

Federal Departement of Home Affairs FDHA
Federal Office of Meteorology and Climatology MeteoSwiss

Kraehbuehlstrasse 58, P.O. Box 514, CH-8044 Zurich, Switzerland

Tel. +41 44 256 93 59
Fax +41 44 256 92 78
oliver.fuhrer@meteoswiss.ch
www.meteoswiss.ch - First-hand information
  
_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Mon Jul 18 00:37:12 2011

This archive was generated by hypermail 2.1.8 : Tue Aug 02 2011 - 14:31:37 MDT