RE: [ncl-talk] NCL operationally

Hi Mary,

I've made a *.tar.gz of our "benchmark" and put it on our FTP server.
You can access it via anonymous login under
ftp://ftp.cscs.ch/in/put/abc/cosmo/fuo/bench.tar.gz . It contains three
directories. The first is called "gt". It contains our local grib
tables. You will need to set your NCL_GRIB_PTABLE_PATH environment
variable to that directory in order for the example to work. The second
is called "cosmolib". It contains a library of NCL commands we are in
the process of developing, which is quite similar to the library used
for WRF. The third is called "bench" and contains the file test.ncl,
which is the actual benchmark including the grib files used for the
benchmark. Please tell me if you have any troubles executing the script,
or if there is anything you are still missing.

You also asked me about our machine details, so that you could send me a
ctrans binary. I work on the following platform...

> uname -a
Linux buin1 2.6.5-7.283-ss #4 SMP Fri Sep 28 13:24:48 PDT 2007 x86_64
x86_64 x86_64 GNU/Linux
> gcc --version
gcc (GCC) 4.2.3 20080201 (rpm:2)

Kind regards,
Oli

________________________________________

Oliver Fuhrer
Numerical Models

Federal Departement of Home Affairs FDHA
Federal Office of Meteorology and Climatology MeteoSwiss

Kraehbuehlstrasse 58, P.O. Box 514, CH-8044 Zurich, Switzerland

Tel. +41 44 256 93 59
Fax +41 44 256 92 78
oliver.fuhrer@meteoswiss.ch
www.meteoswiss.ch - First-hand information
  
 

> -----Original Message-----
> From: Mary Haley [mailto:haley@ucar.edu]
> Sent: Freitag, 23. Januar 2009 00:25
> To: Fuhrer Oliver
> Cc: ncl-talk@ucar.edu
> Subject: RE: [ncl-talk] NCL operationally
>
>
> Hi Oliver,
>
> Thanks for the valuable feedback here. I will address some of your
> points below, and then hopefully Dave Brown, our tech lead, will
> address some other points (and/or correct my own), especially the ones
> on GRIB files.
>
> Please see my comments interspersed.
>
> On Thu, 22 Jan 2009 Oliver.Fuhrer@meteoswiss.ch wrote:
>
> > Hi all,
> >
> > This is just a follow up on my from January 16 concerning a
> comparison
> > of NCL vs. MetView. We've continued the testing and
> improved quite a lot
> > the performance NCL. Our benchmark is a script which
> produces 3 plots
> > (RH @ 850hPa, T + Geopot. Height @ 850Hpa, ThetaE + Geopot. Height @
> > 850hPa). The equivalent potential temperature is computed from model
> > variables in the graphics package. We repeat the script 10x
> and write
> > output to *.eps in NCL which is afterwards converted to *.png with
> > convert or pstoimg. MetView can write directly png files
> and thus the
> > values for the conversion are missing. The results in seconds are as
> > follows...
> >
> > -------------------------------------------
> > PS Plot Conversion to png Total
> > convert pstoimg
> > -------------------------------------------
> > MetView (46) 50
> > NCL 28 26 18 46
> > -------------------------------------------
>
> I sure hope we can get a good PNG driver out. I'm not sure why the
> experimental one is so slow. What were the times for it?
>
> > Several things to note:
> >
> > - NCL performs badly for reading large (huge) grib files.
> We've used an
> > external program to reduce the number of grib records to what we
> > actually need for plotting.
> >
> > - As mentioned in one the posts, pstoimg outperforms convert, even
> > though it is based on ghostscript under the hood. Probably
> the antialias
> > is done differently.
>
> From the NCL end, does it make a difference if you start with a "ps"
> rather than an "eps" file?
>
> > - NCL would slow down by a factor of 2-3 if it would also
> display the
> > data in a transformed projection. We use the "native grid"
> > (tfDoNDCOverlay = "NDCDataExtent") method in order to drastically
> > improve performance. Since MetView is displaying the data in a
> > Stereographic projection and our model uses a CylindricalEquidistant
> > projection with a rotated grid, the comparison is not
> completely fair. I
> > suppose this is the price we pay for having the option of
> "on the fly"
> > transformation of NCARG Graphics the moment the plots are overlaid!
>
> This makes sense. The transformation can be expensive.
>
> > - The experimental PNG driver seems to be very slow (as
> already noted in
> > a post) and conversion of *.ps or *.pdf seems to be the way to go.
> > Nevertheless, I think there is considerable interest in the
> community to
> > have fast and efficient *.png output from NCL.
>
> It would be of interest to try to figure out the reason for the
> slowness of the PNG driver.
>
> Would it be possible for you to provide us with all the data files and
> scripts so we can look at things here? You can email me offline about
> this.
>
> > - We will probably start introducing NCL operationally for
> the first few
> > plots quite soon... :-)
>
> Glad to hear that. :-)
>
> > - I would have liked to try the option of using the NCAR Graphics
> > Metafile format and conversion to *.png via ctrans. But
> ctrans does not
> > seem to be part of my NCL distribution. At least it does
> not sit in the
> > $NCARG_ROOT/bin directory along with idt, ncl, and all the
> rest... Any
> > ideas?
>
> It should be there, as we include ctrans with all distributions of
> NCL. Did you build from source code? If so, perhaps ctrans didn't
> build successfully.
>
> If you want, you can email me offline about your machine information
> (uname -a, gcc --version) and I'll get you a ctrans binary if
> possible.
>
> However, you may find that the NCGM->raster file->PNG route doesn't
> produce as nice of an image. It depends partly on what fonts you're
> using. Also, going the ctrans route seems like an unnecessarily
> expensive step. I usually encourage people to not go this route.
>
> > Thanks for all the posts and help!
>
> And thanks for the update. It's of great interest to us to hear how
> people are using NCL, and where they are finding bottlenecks.
>
> --Mary
>
> > Cheers,
> > Oli
> >
> > ________________________________________
> >
> > Oliver Fuhrer
> > Numerical Models
> >
> > Federal Departement of Home Affairs FDHA
> > Federal Office of Meteorology and Climatology MeteoSwiss
> >
> > Kraehbuehlstrasse 58, P.O. Box 514, CH-8044 Zurich, Switzerland
> >
> > Tel. +41 44 256 93 59
> > Fax +41 44 256 92 78
> > oliver.fuhrer@meteoswiss.ch
> > www.meteoswiss.ch - First-hand information
> >
> >
> >
> >> -----Original Message-----
> >> From: ncl-talk-bounces@ucar.edu
> >> [mailto:ncl-talk-bounces@ucar.edu] On Behalf Of
> >> Oliver.Fuhrer@meteoswiss.ch
> >> Sent: Montag, 19. Januar 2009 16:09
> >> To: ncl-talk@ucar.edu
> >> Subject: NCL operationally
> >>
> >> Hi all,
> >>
> >> We are currently benchmarking NCL against MetView (from
> >> ECMWF). The goal would be to use NCL operationally to produce
> >> plots of our model output. Most of these products are
> >> produced in time-critical mode.
> >>
> >> Currently, there seem to be two bottlenecks that are slowing
> >> NCL down considerably as compared to MetView. The first is
> >> the reading of our model output in grib1 format. After
> >> offline conversion using ncl_convert2nc to NetCDF format and
> >> execution of the same script simply reading a couple of
> >> fields from a couple of files, the script runs 10x faster.
> >> The second is the generation of raster graphics using NCL. A
> >> lot of our plots are disseminated in PNG or GIF format.
> >> MetView can directly write these formats whereas NCL writes a
> >> PS or PDF and the raster format is produced using the
> >> "convert" utility.
> >>
> >> 1) I would be very much interested in future plans for NCL
> >> concerning these two points.
> >>
> >> 2) Would a generation of NCGM and conversion using ctrans to
> >> a raster format bring any gain in performance?
> >>
> >> Thanks and kind regards,
> >> Oliver
> >>
> >>
> >> ________________________________________
> >>
> >> Oliver Fuhrer
> >> Numerical Models
> >>
> >> Federal Departement of Home Affairs FDHA
> >> Federal Office of Meteorology and Climatology MeteoSwiss
> >>
> >> Kraehbuehlstrasse 58, P.O. Box 514, CH-8044 Zurich, Switzerland
> >>
> >> Tel. +41 44 256 93 59
> >> Fax +41 44 256 92 78
> >> oliver.fuhrer@meteoswiss.ch
> >> www.meteoswiss.ch - First-hand information
> >>
> >>
> >>
> >>
> >>
> > _______________________________________________
> > ncl-talk mailing list
> > List instructions, subscriber options, unsubscribe:
> > http://mailman.ucar.edu/mailman/listinfo/ncl-talk
> >
>
Received on Mon Jan 26 10:32:26 2009