Re: ncl_filedump and wgrib

From: Gus Correa <gus_at_nyahnyahspammersnyahnyah>
Date: Thu Sep 29 2011 - 16:28:01 MDT

Hi David, Ahmed

I came across to this type of problem in the past, with the CFS model
output [grib1 files].
I used ncl_convert2nc [which seems to behave similar to ncl_filedump
used by Ahmed] to convert the data from grib1 to NetCDF,
and in some occasions there was more than one variable in the
output for a single variable in the input file [just as in Ahmed's case].

There was yet another discussion about this problem started by
Allison Steiner in this list last July.
See this thread, it may be helpful:

http://www.ncl.ucar.edu/Support/talk_archives/2011/1795.html

However, in that case [and in mine] the problem was related to
time grid variation, because models [CFS, CSFR, etc] seem to
do something about the time grid (not the spatial grids)
that doesn't make Grib very happy.

I am not savvy on grib format, but I suppose grib was
designed to acommodate data that might have grids
with varying spatial resolution, which might also have
variable time sampling interval, etc.
It was designed particularly for observational data,
where stations or other observation platforms may fail, new ones may
come to play, some may sample once a day, others every hour, etc,
hence producing sparse and irregular data.

By contrast NetCDF, maybe other formats, have a more stable format,
often handling model data,
with grid sizes declared ahead of time [except for the time/record
dimension].
Maybe NCL adopts an approach similar to NetCDF, expecting grids to
be static, regular.
I suppose that following this logic,
NCL splits any Grib irregular grid in a
minimal set of regular grids, which is what puzzles Ahmed,
but was clearly explained by David in the case of the longitude grid.

Someone correct me if I am wrong, but grib seems to store
the spatial grids for each time record, whereas
NetCDF stores a single set of spatial grids for all time records.

In the past I was also distressed by NCL producing several variables for
a single grib variable on the input, whenever any of the grids change.
However, now I am convinced that NCL does the right thing, as David said.

Now, here is where I don't agree with David.
If Ahmed data is observational, there may be no mistake
whatsoever during its production.
The irregular spatial grids may just reflect the irregularity
of the sampling process in the real world.

My two cents,
Gus Correa

David Brown wrote:
> Hi Ahmed,
>
> I examined your file and have found that the variables you list below
> differ not only in time but also in space: there are 2 different grids
> represented. They both have the same latitude elements, but in
> longitude, one grid has 277 elements from -16.686 to 75.22 while the
> other has 261 elements from -16.686 to 70. You see this with wgrib if
> you run it as wgrib -V and compare, for instance, record 167 and 168
> (both APCP in wgrib). Considering the randomness of the way the 2 grids
> are interspersed in the forecast time sequence, perhaps there was a
> mistake in the production of these files, but NCL has no choice but to
> consider these as two different variables. As for the variables with the
> suffix _acc, they do belong in some sense to the corresponding variable
> with the _acc3h suffix: they are the initial record at the beginning of
> the accumulation period. However, since no time has passed they do not
> fit the 3 hour accumulation designation. Their values are all either 0
> or the missing value so their absence from the time array dimension
> should be easy to work around.
> -dave
>
> On Sep 28, 2011, at 4:15 PM, ahmed lasheen wrote:
>
>> thanks for your reply .
>> i read the NCL manual for how NCL process the grib file.But i
>> still wondering for example if NCL finds that the grib file contains
>> several records with the same variable name ,with the same level
>> (ex: isobaric level) and same dimension , differ only in time , then
>> why it nesed to create different variables .the algorithm of the NCL
>> as i understand should not permit generation of such variables.
>> in my grib file i have 3 variables only and here is the dump of the
>> file using the ncl.
>> sometimes these variables are misleading and confusing.
>> float A_PCP_GDS0_SFC_acc ( g0_lat_0, g0_lon_1 )
>> float A_PCP_GDS0_SFC_acc3h ( forecast_time2, g0_lat_0, g0_lon_1)
>> float A_PCP_GDS0_SFC_acc3h_1 ( forecast_time3, g0_lat_4, g0_lon_5 )
>> float NCPCP_GDS0_SFC_acc ( g0_lat_0, g0_lon_1 )
>> float NCPCP_GDS0_SFC_acc3h ( forecast_time2, g0_lat_0, g0_lon_1 )
>> float NCPCP_GDS0_SFC_acc3h_1 ( forecast_time3, g0_lat_4, g0_lon_1)
>> float ACPCP_GDS0_SFC_acc ( g0_lat_0, g0_lon_1 )
>> float ACPCP_GDS0_SFC_acc3h ( forecast_time2, g0_lat_0, g0_lon_1
>> float ACPCP_GDS0_SFC_acc3h_1 ( forecast_time3, g0_lat_4, g0_lon_5 )
>> i aslo have attached the dump of file using both wgrib and ncl_fildump
>>
>> On Wed, Sep 28, 2011 at 11:18 PM, David Brown <dbrown@ucar.edu
>> <mailto:dbrown@ucar.edu>> wrote:
>>
>> Hi Ahmed,
>>
>> wgrib simply lists each record in a GRIB file in the order it
>> finds them. NCL looks through the whole file and organizes the
>> records into variables with time and level dimensions in addition
>> to the horizontal lat/lon dimensions. But in order to add those
>> dimensions, it needs to make sure that the records are
>> compatible. GRIB has many types of level records for instance
>> (e.g. sigma levels, hybrid levels, pressure levels). In order to
>> create a valid level dimension, all the records used must have the
>> same type of level, in addition to representing the same type of
>> quantity. Further, the GRIB file can contain records with the same
>> type of quantity but on different horizontal grids, with different
>> extents or resolutions or projection type. These must be
>> represented with different variables. The whole process that NCL
>> uses to represent GRIB data is described in detail
>> at http://www.ncl.ucar.edu/Document/Manuals/Ref_Manual/NclFormatSupport.shtml#GRIB.
>> I hope this helps.
>> -dave
>>
>>
>> On Sep 28, 2011, at 1:31 PM, ahmed lasheen wrote:
>>
>>> hello
>>> when i make wgrib for grib1 file contains precipitation data , i
>>> found three variables which are precipitation of grid scale
>>> , precipitation of convective scale and total
>>> scale precipitation.But when i make ncl_filedump i see 9
>>> variables which represent those variables plus others
>>> related variables to them
>>> but different on the average time and grid location.why this
>>> occur ? and what this difference means between wgrib and
>>> ncl_filedump?.
>>> thanks in advance
>>>
>>> --
>>> ===============
>>> Ahmed Lasheen
>>> Junior researcher at Cairo Numerical Weather Prediction Center
>>> (CNWPC)
>>> Egyptian Meteorological Authority(EMA)
>>> Cairo,Egypt
>>> ===============
>>>
>>> _______________________________________________
>>> ncl-talk mailing list
>>> List instructions, subscriber options, unsubscribe:
>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>
>>
>>
>>
>> --
>> ===============
>> Ahmed Lasheen
>> Junior researcher at Cairo Numerical Weather Prediction Center (CNWPC)
>> Egyptian Meteorological Authority(EMA)
>> Cairo,Egypt
>> ===============
>>
>> <wgrib><ncl>_______________________________________________
>> ncl-talk mailing list
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Thu Sep 29 16:28:14 2011

This archive was generated by hypermail 2.1.8 : Sun Oct 09 2011 - 13:05:26 MDT