Re: ncl performance

Hi Dennis,

This might be a detail, but I think NCL is doing some multiprocessing...
We were benchmarking NCL for some time and investigating on how to use
it optimally on our dual core processors. Doing that we noted that NCARG
actually uses the "fork" command when finishing up graphics and thus
sometimes has 2 or even more (depending on how often your script is
finishing up a graphics file) parallel jobs running all using the same
amount of memory.

This feature was actually rather annoying to us as it sometimes led the
machine into swapping.

Cheers,
Oli

 
________________________________________

Oliver Fuhrer
Numerical Models

Federal Departement of Home Affairs FDHA
Federal Office of Meteorology and Climatology MeteoSwiss

Kraehbuehlstrasse 58, P.O. Box 514, CH-8044 Zurich, Switzerland

Tel. +41 44 256 93 59
Fax +41 44 256 92 78
oliver.fuhrer_at_meteoswiss.ch
www.meteoswiss.ch - First-hand information
  
 

> -----Original Message-----
> From: ncl-talk-bounces_at_ucar.edu
> [mailto:ncl-talk-bounces_at_ucar.edu] On Behalf Of Dennis Shea
> Sent: Montag, 15. Juni 2009 18:37
> To: Micah Sklut
> Cc: NCL_Talk
> Subject: Re: ncl performance
>
> Multithreading or multiprocessing or parallel processing??
>
> AFAIK multithreading is more system level. I don't think this
> is user controllable.
>
> Multiprocessing: It is not done from the NCL level. The
> user can spawn multiple jobs to run on different CPUs from
> (say) a C-shell script. Of course, if you have only one
> data channel per cpu to a common memory, the different processes
> will still have IO 'conflicts' when reading/writing to a common
> (shared) memory.
>
> Parallel processing: Requires 'load balancing'' to work well.
> Some algorithms [eg: eigenvector conputations] could be set
> up to use this and (possibly) some graphics
> calculations but, currently, NCL does not include this capability.
>
> D
>
>
>
>
>
> Micah Sklut wrote:
> > Dennis,
> >
> > Can NCL utilize multi threading? or is it only going to use one
> > processor?
> >
> >
> > On Fri, Jun 12, 2009 at 6:47 PM, Dennis Shea <shea_at_ucar.edu
> > <mailto:shea_at_ucar.edu>> wrote:
> >
> > IMHO ... memory is always good to have ...
> >
> > Often mhz are emphasized ... not that important in and of
> > itself. Overlooked issues are data movement from memory
> and cache.
> >
> > You can have lots of memory and a fast cpu but if the data
> > movement is slow then the code will run slowly. I do not
> > just mean reading from memory ... writing takes time also.
> > If you have intertwined read/write tasks then performance
> > could be even slower.
> >
> > D
> >
> > Micah Sklut wrote:
> >
> > Hi NCL,
> >
> > Its been a while, since my last questions, but I believe I
> > have a straight forward one here.
> >
> > What is the best way to increase performance for
> ncl scripts.
> > I have batch jobs where I read in large grib files,
> and output
> > lots of images.
> >
> > Will performance improve by increasing RAM ?
> > Will performance improve by increasing Processor Speed?
> >
> > My guess, is that the memory is going to be the biggest
> > factor, and then processor speed second.
> >
> > Thanks for your help!
> >
> > Micah
> >
> --------------------------------------------------------------
> ----------
> >
> > _______________________________________________
> > ncl-talk mailing list
> > List instructions, subscriber options, unsubscribe:
> > http://mailman.ucar.edu/mailman/listinfo/ncl-talk
> >
> >
> >
> >
> > --
> > ======================================================
> > Dennis J. Shea tel: 303-497-1361 |
> > P.O. Box 3000 fax: 303-497-1333 |
> > Climate Analysis Section |
> > Climate & Global Dynamics Div. |
> > National Center for Atmospheric Research |
> > Boulder, CO 80307 |
> > USA email: shea 'at' ucar.edu
> > <http://ucar.edu> |
> > ======================================================
> >
> >
>
>
> --
> ======================================================
> Dennis J. Shea tel: 303-497-1361 |
> P.O. Box 3000 fax: 303-497-1333 |
> Climate Analysis Section |
> Climate & Global Dynamics Div. |
> National Center for Atmospheric Research |
> Boulder, CO 80307 |
> USA email: shea 'at' ucar.edu |
> ======================================================
>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Mon Jun 15 2009 - 23:52:34 MDT