help with reading grib files correctly --> setfileoption("grb", "TimePeriodSuffix", False)

From: Erik Noble <nobleeu_at_nyahnyahspammersnyahnyah>
Date: Mon Sep 26 2011 - 11:48:09 MDT

Dear NCL,
I don't know exactly where to begin with this.
 I am having trouble using the setfileoption & TimePeriodSuffix option to
read grib data from the WRF UPP post-proccessor. I initially contacted
WRFhelp about this; after several emails, WRFhelp suggested I contact NCL to
resolve this. ( The conversation is below. The script I am using is below
too.)

Basically, I am using ncl's "addfiles" function to read in several variables
from 96 grib files. Everything works just fine, up to the 86th file, where
the name of the variable, regardless of the variable I ask for, has a "_10"
appended on the end of the variable name. For example, I need to plot U and
V winds at isobaric levels. When I try to extract the U variable, which
reads as U_GRD_GDS1_ISBL from ncl_filedump, I notice that after
forecast hour 258 (from wrfout file 2006-09-12-18) it "changes to a new
name, "U_GRD_GDS1_ISBL_10."

The WRFhelp team claims this is an NCL issue but at the same time suggested
I try using the setfileoption & TimePeriodSuffix. I still get the same
result, the last 10 files will not read.

How do I resolve this. Further details towards my converstaion with wrfHelp
and my code is below.
Thank you in advance for your help.
-Erik Noble

On Sep 26, 2011, at 10:29 AM, WRFHelp wrote:

Hey Erik-

I have to admit I have never used setfileoption. But I would first
check -
Are value 86+ still in the variable with the _10 appended to the end?
If so
you can fire a question to ncl_help and ask why. You may need to read both
values and swap at the byte roll over time if NCL doesn't support the
suppression
of renaming the variable. If you think the data in the
unipost grib output file is wrong please send me at least one input
and output
file (preferably fhr 258 or beyond). You are correct to send me (FTP)
the file you will
have to rename it removing the ":", sorry.

If you get this working with NCL I would be interested in how you
achieved it. Thanks
and best of luck on the journey.

--wrfhelp

On Fri, 23 Sep 2011 18:26:54 -0500 "Noble, Erik U. (GSFC)[COLUMBIA
UNIVERSITY]" <erik.u.noble@nasa.gov>
wrote:

Hi. Ok. I see. I followed your suggestion.
I used the set file option
setfileoption("grb","TimePeriodSuffix",False) before reading in the
files, as the link suggests.

But, the scripts stopped at the 85th file. There are 96 files (or
rather 96 values that are supposed to be there).
Is there something I am doing wrong?

NCL Script

;***************************************************************
; Load Libaries
;***************************************************************
load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"
load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"
load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"
load "$NCARG_ROOT/lib/ncarg/nclscripts/wrf/WRF_contributed.ncl"

begin
;***************************************************************
; Change these parts
;***************************************************************
DATADir = "./" ;input directory
Files = "WRFPRS_*" ;input files
ncdiro = "./" ;output directory
NCFILE1 = "pressure" ;output file
;***************************************************************
; Read in WRF Files
;***************************************************************
FILES = systemfunc (" ls " + Files) ; file paths
numFILES = dimsizes(FILES)
print(numFILES)

setfileoption("grb","TimePeriodSuffix",False)

files = addfiles (FILES+".grb", "r")
;========================
 ListSetType (files, "join") ; concatenate "cat" or merge
"join"
; z = files[:]->HGT_GDS1_ISBL ;Geopotential height
; t = files[:]->TMP_GDS1_ISBL
; td = files[:]->DPT_GDS1_ISBL
 u = files[:]->U_GRD_GDS1_ISBL
; v = files[:]->V_GRD_GDS1_ISBL
; sph = files[:]->SPF_H_GDS1_ISBL
; rh = files[:]->R_H_GDS1_ISBL

printVarSummary(u)
printMinMax(u,True)
print(u(:,0,0,0))
end

Terminal Output
[eunoble@discover18] post_Experiment_01 $ ncl test_pressure.ncl
Copyright (C) 1995-2011 - All Rights Reserved
University Corporation for Atmospheric Research
NCAR Command Language Version 6.0.0
The use of this software is governed by a License Agreement.
See http://www.ncl.ucar.edu/ for more details.

Variable: numFILES
Type: integer
Total Size: 4 bytes
          1 values
Number of Dimensions: 1
Dimensions and sizes: [1]
Coordinates:
(0) 97

Variable: u
Type: float
Total Size: 1051117800 bytes
          262779450 values
Number of Dimensions: 4
Dimensions and sizes: [ncl_join | 86] x [lv_ISBL3 | 25] x [g1_lat_0
| 311] x [g1_lon_1 | 393]
Coordinates:
          lv_ISBL3: [50..1000]
          g1_lat_0: [-18.402..35.335]
          g1_lon_1: [324.251..35.749]
Number Of Attributes: 12
center : US National Weather Service - NCEP (WMC)
long_name : u-component of wind
units : m/s
_FillValue : 1e+20
level_indicator : 100
gds_grid_type : 1
parameter_table_version : 2
parameter_number : 33
model : North Pacific Hurricane Wave Model
forecast_time : 0
forecast_time_units : hours
initial_time : 09/02/2006 (00:00)
(0)
(0) u-component of wind: min=-58.0981 max=48.6898

Variable: u (subsection)
Type: float
Total Size: 344 bytes
          86 values
Number of Dimensions: 1
Dimensions and sizes: [ncl_join | 86]
Coordinates:
Number Of Attributes: 15
g1_lon_1 : 324.251
g1_lat_0 : -18.402
lv_ISBL3 : 50
initial_time : 09/02/2006 (00:00)
forecast_time_units : hours
forecast_time : 0
model : North Pacific Hurricane Wave Model
parameter_number : 33
parameter_table_version : 2
gds_grid_type : 1
level_indicator : 100
_FillValue : 1e+20
units : m/s
long_name : u-component of wind
center : US National Weather Service - NCEP (WMC)
(0) -0.2010425
(1) 0.5233667
(2) 1.252354
(3) 2.798071
(4) 4.349082
(5) 2.420659
(6) 0.4866016
(7) 1.312463
(8) 2.137537
(9) 3.360134
(10) 4.575947
(11) 5.127134
(12) 5.67635
(13) 5.544646
(14) 5.411767
(15) 5.634192
(16) 5.861868
(17) 6.084302
(18) 6.315991
(19) 5.298777
(20) 4.281072
(21) 3.456113
(22) 2.63082
(23) 2.895886
(24) 3.170952
(25) 4.251602
(26) 5.347261
(27) 4.80581
(28) 4.269766
(29) 4.413843
(30) 4.56592
(31) 4.179683
(32) 3.794978
(33) 4.186904
(34) 4.583442
(35) 4.875342
(36) 5.170044
(37) 4.927932
(38) 4.673596
(39) 4.54365
(40) 4.401719
(41) 5.855395
(42) 7.312293
(43) 8.062078
(44) 8.802468
(45) 7.902993
(46) 7.010733
(47) 6.265562
(48) 5.535437
(49) 6.774873
(50) 8.025661
(51) 8.683907
(52) 9.33523
(53) 8.577312
(54) 7.812954
(55) 7.792734
(56) 7.772768
(57) 7.932007
(58) 8.090359
(59) 8.378562
(60) 8.665559
(61) 6.949731
(62) 5.23176
(63) 7.035208
(64) 8.830354
(65) 8.120037
(66) 7.405535
(67) 7.409487
(68) 7.408035
(69) 6.654251
(70) 5.898555
(71) 6.32137
(72) 6.728426
(73) 7.347017
(74) 7.962986
(75) 6.482971
(76) 5.008667
(77) 4.458303
(78) 3.911145
(79) 4.192146
(80) 4.46812
(81) 4.676645
(82) 4.890271
(83) 5.270593
(84) 5.647607
(85) 5.126626

On Sep 23, 2011, at 10:25 AM, WRF Help wrote:

Excellent --

OK -- why? Well the GRIB format originally only had 8 bits for
representing the forecast hours. So you ran out of numbers at 255
(decimal). Well today in the GRIB PDS we set byte 21 (1 relative) to 10 to
indicate we need to use bytes 19 AND 20 to represent P1 (the time). So in
your UPP GRIB output at forecast hour 258 the time representation in the
GRIB file changes. NCL looks at these
values and when it sees the time representation change it changes the
variable name. See
http://www.ncl.ucar.edu/Document/Manuals/Ref_Manual/NclFormatSupport.shtml#GRIB
for more details about that. Unfortunately in your case this is not a new
variable just a format dependency.

Now I have never tried this but it sounds like it might work. In NCL use
the setfileoption function to turn off the TimePeriodSuffix (not available
in NCL until 5.1.0).

Hope that helps -- otherwise assume both names are the same variable.

--wrfhelp

On Wed, 21 Sep 2011 17:57:24 -0500 "Noble, Erik U. (GSFC)[COLUMBIA
UNIVERSITY]" <erik.u.noble@nasa.gov<mailto:erik.u.noble@nasa.gov>> wrote:

Dear WRF HELP,
I have 96 WRF output files (every 3 hours, 288 forecast hours total.)
I ran unipost in your UPP. It created the files WRFPRS_d01.000 to
WRFPRS_d01.288. So far so good.

But when I actually try look at the new data itself, using NCL's
ncl_filedump command, the last 10 WRFPRS
files are missing data.
The only way I can prove this is to show you the results from the
ncl_filedump command script that prints out the value for every single
time-step. The last 10 files are missing, but the actual WRF output files
are full of data. What happened?

The variables change.

I need to plot U and V winds at isobaric levels. When I try to
extract the variable I need, U_GRD_GDS1_ISBL, I notice that after
forecast hour 258 (from wrfout file 2006-09-12-18) it "changes to a
new name, "U_GRD_GDS1_ISBL_10."
It is not that the last 10 files are missing data, it is that the
variable names have changed. Why would this occur?
See below.
-Erik

NCL Script

;***************************************************************
; Load Libaries
;***************************************************************
load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"
load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"
load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"
load "$NCARG_ROOT/lib/ncarg/nclscripts/wrf/WRF_contributed.ncl"

begin
;***************************************************************
; Change these parts
;***************************************************************
DATADir = "./" ;input directory
Files = "WRFPRS_*" ;input files
ncdiro = "./" ;output directory
NCFILE1 = "pressure" ;output file
;***************************************************************
; Read in WRF Files
;***************************************************************
FILES = systemfunc (" ls " + Files) ; file paths
numFILES = dimsizes(FILES)
print(numFILES)

setfileoption("grb","TimePeriodSuffix",False)

files = addfiles (FILES+".grb", "r")
;========================
 ListSetType (files, "join") ; concatenate "cat" or merge
"join"
; z = files[:]->HGT_GDS1_ISBL ;Geopotential height
; t = files[:]->TMP_GDS1_ISBL
; td = files[:]->DPT_GDS1_ISBL
 u = files[:]->U_GRD_GDS1_ISBL
; v = files[:]->V_GRD_GDS1_ISBL
; sph = files[:]->SPF_H_GDS1_ISBL
; rh = files[:]->R_H_GDS1_ISBL

printVarSummary(u)
printMinMax(u,True)
print(u(:,0,0,0))
end

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Mon Sep 26 11:48:19 2011

This archive was generated by hypermail 2.1.8 : Mon Sep 26 2011 - 14:45:50 MDT