RE: [ncl-talk] NCL operationally

Hi Oliver,

Thanks for the valuable feedback here. I will address some of your
points below, and then hopefully Dave Brown, our tech lead, will
address some other points (and/or correct my own), especially the ones
on GRIB files.

Please see my comments interspersed.

On Thu, 22 Jan 2009 Oliver.Fuhrer_at_meteoswiss.ch wrote:

> Hi all,
>
> This is just a follow up on my from January 16 concerning a comparison
> of NCL vs. MetView. We've continued the testing and improved quite a lot
> the performance NCL. Our benchmark is a script which produces 3 plots
> (RH @ 850hPa, T + Geopot. Height @ 850Hpa, ThetaE + Geopot. Height @
> 850hPa). The equivalent potential temperature is computed from model
> variables in the graphics package. We repeat the script 10x and write
> output to *.eps in NCL which is afterwards converted to *.png with
> convert or pstoimg. MetView can write directly png files and thus the
> values for the conversion are missing. The results in seconds are as
> follows...
>
> -------------------------------------------
> PS Plot Conversion to png Total
> convert pstoimg
> -------------------------------------------
> MetView (46) 50
> NCL 28 26 18 46
> -------------------------------------------

I sure hope we can get a good PNG driver out. I'm not sure why the
experimental one is so slow. What were the times for it?

> Several things to note:
>
> - NCL performs badly for reading large (huge) grib files. We've used an
> external program to reduce the number of grib records to what we
> actually need for plotting.
>
> - As mentioned in one the posts, pstoimg outperforms convert, even
> though it is based on ghostscript under the hood. Probably the antialias
> is done differently.

>From the NCL end, does it make a difference if you start with a "ps"
rather than an "eps" file?

> - NCL would slow down by a factor of 2-3 if it would also display the
> data in a transformed projection. We use the "native grid"
> (tfDoNDCOverlay = "NDCDataExtent") method in order to drastically
> improve performance. Since MetView is displaying the data in a
> Stereographic projection and our model uses a CylindricalEquidistant
> projection with a rotated grid, the comparison is not completely fair. I
> suppose this is the price we pay for having the option of "on the fly"
> transformation of NCARG Graphics the moment the plots are overlaid!

This makes sense. The transformation can be expensive.

> - The experimental PNG driver seems to be very slow (as already noted in
> a post) and conversion of *.ps or *.pdf seems to be the way to go.
> Nevertheless, I think there is considerable interest in the community to
> have fast and efficient *.png output from NCL.

It would be of interest to try to figure out the reason for the
slowness of the PNG driver.

Would it be possible for you to provide us with all the data files and
scripts so we can look at things here? You can email me offline about
this.

> - We will probably start introducing NCL operationally for the first few
> plots quite soon... :-)

Glad to hear that. :-)

> - I would have liked to try the option of using the NCAR Graphics
> Metafile format and conversion to *.png via ctrans. But ctrans does not
> seem to be part of my NCL distribution. At least it does not sit in the
> $NCARG_ROOT/bin directory along with idt, ncl, and all the rest... Any
> ideas?

It should be there, as we include ctrans with all distributions of
NCL. Did you build from source code? If so, perhaps ctrans didn't
build successfully.

If you want, you can email me offline about your machine information
(uname -a, gcc --version) and I'll get you a ctrans binary if
possible.

However, you may find that the NCGM->raster file->PNG route doesn't
produce as nice of an image. It depends partly on what fonts you're
using. Also, going the ctrans route seems like an unnecessarily
expensive step. I usually encourage people to not go this route.

> Thanks for all the posts and help!

And thanks for the update. It's of great interest to us to hear how
people are using NCL, and where they are finding bottlenecks.

--Mary

> Cheers,
> Oli
>
> ________________________________________
>
> Oliver Fuhrer
> Numerical Models
>
> Federal Departement of Home Affairs FDHA
> Federal Office of Meteorology and Climatology MeteoSwiss
>
> Kraehbuehlstrasse 58, P.O. Box 514, CH-8044 Zurich, Switzerland
>
> Tel. +41 44 256 93 59
> Fax +41 44 256 92 78
> oliver.fuhrer_at_meteoswiss.ch
> www.meteoswiss.ch - First-hand information
>
>
>
>> -----Original Message-----
>> From: ncl-talk-bounces_at_ucar.edu
>> [mailto:ncl-talk-bounces_at_ucar.edu] On Behalf Of
>> Oliver.Fuhrer_at_meteoswiss.ch
>> Sent: Montag, 19. Januar 2009 16:09
>> To: ncl-talk_at_ucar.edu
>> Subject: NCL operationally
>>
>> Hi all,
>>
>> We are currently benchmarking NCL against MetView (from
>> ECMWF). The goal would be to use NCL operationally to produce
>> plots of our model output. Most of these products are
>> produced in time-critical mode.
>>
>> Currently, there seem to be two bottlenecks that are slowing
>> NCL down considerably as compared to MetView. The first is
>> the reading of our model output in grib1 format. After
>> offline conversion using ncl_convert2nc to NetCDF format and
>> execution of the same script simply reading a couple of
>> fields from a couple of files, the script runs 10x faster.
>> The second is the generation of raster graphics using NCL. A
>> lot of our plots are disseminated in PNG or GIF format.
>> MetView can directly write these formats whereas NCL writes a
>> PS or PDF and the raster format is produced using the
>> "convert" utility.
>>
>> 1) I would be very much interested in future plans for NCL
>> concerning these two points.
>>
>> 2) Would a generation of NCGM and conversion using ctrans to
>> a raster format bring any gain in performance?
>>
>> Thanks and kind regards,
>> Oliver
>>
>>
>> ________________________________________
>>
>> Oliver Fuhrer
>> Numerical Models
>>
>> Federal Departement of Home Affairs FDHA
>> Federal Office of Meteorology and Climatology MeteoSwiss
>>
>> Kraehbuehlstrasse 58, P.O. Box 514, CH-8044 Zurich, Switzerland
>>
>> Tel. +41 44 256 93 59
>> Fax +41 44 256 92 78
>> oliver.fuhrer_at_meteoswiss.ch
>> www.meteoswiss.ch - First-hand information
>>
>>
>>
>>
>>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Thu Jan 22 2009 - 16:24:49 MST