>
> Hi,
>
> Initially I was extracting 4 variables from Eta 12 km grib files, now I'm
> trying to extract 11 additional variables. I
> editted the script, and it worked, but the run time went from ~1 minute to
> ~30 minutes per job. Here's the code -
>
> ; Set the variables to be converted to netcdf
> pnames=(/"PRES","TMP","U_GRD","V_GRD","ULWRF","USWRF","DLWRF","DSWRF","SHTFL","LHTFL","WTMP","T_CDC","A_PCP","R_H","PRMSL"/)
>
> plevels=(/"SFC","HTGL","HTGL","HTGL","SFC","SFC","SFC","SFC","SFC","SFC","SFC","EATM","SFC_acc3h","HTGL","MSL"/)
>
> ;pvalues=(/1,11,33,34,/)
> ncnames=(/"press","temp","uwind","vwind","ulwrf","uswrf","dlwrf","dswrf","shtfl","lhtfl","wtmp","tcdc","apcp","rh","prmsl"/)
>
> ; Special variables of lat, lon no plevel segment
> vlat= "gridlat_"+gnum
> netcdf_out->lat = grib_in->\$vlat\$;
> vlon= "gridlon_"+gnum
> netcdf_out->lon = grib_in->\$vlon\$;
>
> ; loop looking for matching variable names
> do j=0,dimsizes(ncnames)-1
> vnamej=pnames(j)+"_"+gnum+"_"+plevels(j)
> do i=0,nNames-1
> ; test for vnamej=grib_in->\$vNames(i)\$
> if (vnamej.eq.vNames(i)) then
> print ("Match "+j+" "+i+" "+vnamej+" "+vNames(i)+"
> "+ncnames(j))
> netcdf_out->\$ncnames(j)\$ = grib_in->\$vNames(i)\$
> delete(grib_in->\$vNames(i)\$)
> delete(netcdf_out->\$ncnames(j)\$)
> end if
> end do
> end do
>
> I've noticed that the script progressively takes longer to extract the
> variables as it goes down the list. One of my coworkers thought that this
> may be due to each variable taking additional memory. I tried to use
> "delete" and "undef" to clear the variables from memory, but the script
> still took a long time. Any ideas? Thanks.
>
> Zack
Hi Zack,
I strongly recommend taking Sylvia's suggestion of reviewing how to
write netCDF files "efficiently" from the URL she pointed out:
http://www.cgd.ucar.edu/csm/support/FH/method_2.shtml
It may look tedious at first, but pre-defining the variables in this
matter, and especially the coordinate array information, can
significantly improve the speed. If you have any questions about this,
let us know in a private email.
FYI, in your code snippet above, you have:
delete(netcdf_out->\$ncnames(j)\$)
You don't want to do this, because I believe this will delete the
variable from the file you just wrote it to.
Since you are not reading any variables off the file into a local
file, there's no need to do any "delete" calls, so you can also
remove:
delete(grib_in->\$vNames(i)\$)
Part of the slow down is also due to the nested do loops. The more do
loops you have and the more code they contain, the more slow down you
will see. We encourage users to avoid do loops where possible.
If you are not interested in the "efficient" way of writing netCDF
files, then you can trim your two do loops down into one do loop
as follows (please note that this typed off-the-cuff, so there
may be some syntax errors):
...
vlat= "gridlat_"+gnum
netcdf_out->lat = grib_in->\$vlat\$;
vlon= "gridlon_"+gnum
netcdf_out->lon = grib_in->\$vlon\$;
;
; Get list of potential variable names. Note that I'm doing this
; outside the do loop.
;
vnamesF = pnames+"_"+gnum+"_"+plevels
; loop looking for matching variable names
do i=0,nNames-1
; Get index in vnamesF where vNames(i) is equal to one of its values.
; If it's not missing, then we know we've found a match.
index = ind(vNames(i).eq.vnamesF)
if(.not.ismissing(index)) then
netcdf_out->\$ncnames(index)\$ = grib_in->\$vNames(i)\$
end if
end do
_______________________________________________
ncl-talk mailing list
ncl-talk@ucar.edu
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
This archive was generated by hypermail 2b29 : Mon Apr 14 2003 - 15:57:45 MDT