Re: NCL operationally

Arrrrgh .... Somehow my text got messed up:

Dennis Shea wrote:
> Hi Oliver
>
>
> Today is a national holiday. Here in Colorado,
> we are expecting near record breaking tempertures
> [forecast high temperature is 16C :-) ]
> Hence, I do not expect people to be logging in today.
>
> I will let NCL's graphics people answer about future
> png plans.
>
> Comment: the reason why reading the grib1 or grib2
> via NCL can take some time is that NCL is making
> the variables on the GRIB files conform to a netCDF
> data model. The issue is that GRIB is a *record format*
> NOT a *file format* like netCDF or HDF. A GRIB file
> is just a collection of individual grib records. There
> are no rules. The grib records can be in any random
> order. So NCL must look at the *entire file*. On the
> fly, it creates pointers to records, creates uniqu

fly, it creates pointers to records, creates unique
variable names and 'other' tasks to make the
GRIB file's contents

> look like a netCDF file's contents. Importing to this
> common data model comes at a price .... time to organize.
>
> So, I doubt it is the overhead involved with using
> "convert" that is the problem. It is the file
> organization overhead that causes the slowdown.
>
> ---
> I have never used MetView. I speculate it reads
> 'just' the grib records necessary for the plots.
>
> A possible alternative to using ncl_convert2nc offline:
>
> Use wgrib to strip off the required records
> and then have NCL work on the selected subset.
>
> Something along the following:
>
> system('"wgrib -s grib_file | grep ":HGT:500 mb:" | wgrib -i -grib \
> grib_file -o temporary_grib_file" ')
> f = addfile ("temporary_grib_file.grb", "r")
> x = f->HGT...
> [do whatever]
> system("/bin/rm -f temporary_grib_file.grb"
>
>
> See: ftp://ftp.cpc.ncep.noaa.gov/wd51we/wgrib/tricks.wgrib
>
> ----
>
> As far as creating png via convert, my style is the following:
>
> pltDir = "./"
> pltType = "png"
> pltName = "mjo_spectra"
> pltConvert = "-density 96 -trim +repage"
>
> These could be done via NCL's Command Line Arguments if desired
>
> ;------------------------------------------------------------
> ; PLOTS
> ;------------------------------------------------------------
> if (pltType.eq."png") then
> pltTypeLocal = "ps"
> else
> pltTypeLocal = pltType
> end if
>
> pltPath = pltDir+pltName
>
> wks = gsn_open_wks(pltTypeLocal,pltPath)
>
> [plot(s)]
>
> if (pltType.eq."png") then
> if (.not.isvar("pltConvert")) then
> pltConvert = " " ; default
> end if
> system("convert "+pltConvert+" "+pltPath+".ps "+pltPath+".png")
> system("/bin/rm -f "+pltPath+".ps")
> end if
> ========================
>
> Regards
> D
>
> Oliver.Fuhrer_at_meteoswiss.ch wrote:
>> Hi all,
>>
>> We are currently benchmarking NCL against MetView (from ECMWF). The
>> goal would be to use NCL operationally to produce plots of our model
>> output. Most of these products are produced in time-critical mode.
>>
>> Currently, there seem to be two bottlenecks that are slowing NCL down
>> considerably as compared to MetView. The first is the reading of our
>> model output in grib1 format. After offline conversion using
>> ncl_convert2nc to NetCDF format and execution of the same script
>> simply reading a couple of fields from a couple of files, the script
>> runs 10x faster. The second is the generation of raster graphics using
>> NCL. A lot of our plots are disseminated in PNG or GIF format. MetView
>> can directly write these formats whereas NCL writes a PS or PDF and
>> the raster format is produced using the "convert" utility.
>>
>> 1) I would be very much interested in future plans for NCL concerning
>> these two points.
>>
>> 2) Would a generation of NCGM and conversion using ctrans to a raster
>> format bring any gain in performance?
>>
>> Thanks and kind regards,
>> Oliver
>>
>>
>> ________________________________________
>>
>> Oliver Fuhrer
>> Numerical Models
>>
>> Federal Departement of Home Affairs FDHA
>> Federal Office of Meteorology and Climatology MeteoSwiss
>>
>> Kraehbuehlstrasse 58, P.O. Box 514, CH-8044 Zurich, Switzerland
>>
>> Tel. +41 44 256 93 59
>> Fax +41 44 256 92 78
>> oliver.fuhrer_at_meteoswiss.ch
>> www.meteoswiss.ch - First-hand information
>>
>>
>>
>>
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> ncl-talk mailing list
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Mon Jan 19 2009 - 09:23:30 MST