Re: Where function inserting missing values. Why?

Bridget,

Here is another way to look at this. Your original where statement
contained lazy evaluation, and you didn't realize it. The usage was
of this form, "A" being the ismissing function:

   where (A .and. B, true_value, false_value)

Now there is this thing called lazy expression evaluation or short
circuit evaluation. In the most general terms, this means that if
the result can be determined by *only* looking at part A and the
binary operator, then the computer never looks at part B. The
trouble comes over the meaning of "never looks at"!

In NCL, if A and B are *scalars*, and A is False, then lazy
evaluation is used like this:

   False .and. (don't care) = False always

That is what you expected! However if A and B are *arrays*, then
NCL does something different at array positions where B is missing.
  Lazy evaluation is *not* performed!

   False .and. missing = missing

That propagates through the "where" function to missing values in
the function result array. This is what you were originally getting.

This comes up most often when there is a desire to use the
logical-valued "ismissing" function to mask another array, exactly
as you did.

A good solution is to compute the mask very carefully in a separate
where statement, as I showed, and ensure that the mask itself will
not contain ANY missing values.

See previous discussions:

   http://www.ncl.ucar.edu/Support/talk_archives/2009/1203.html
   http://www.ncl.ucar.edu/Support/talk_archives/2009/2175.html

--Dave

Bridget Thrasher wrote:
> Thanks, Dave! That actually worked! But I still don't understand why my
> original statement didn't. What does lazy evaluation have to do with it?
> Does this mean that any complex conditional statements used in a where
> function is not being properly evaluated? For example, I've got the
> following (and others like it) earlier on in my script:
>
> q = where(nzeros20c.gt.(0.95*yrs20c) .and. nzerosobs.lt.(0.4*yrs20c)
> .and. data_all(a*12+m,:,:).gt.0.,q_sh,q)
>
> Thanks again,
> Bridget
>
> Date: Tue, 20 Oct 2009 16:43:15 -0600
> From: Dave Allured <dave.allured@noaa.gov>
>
> Bridget,
>
> Here is something else to try. In the code in your first message,
> try this in place of your last where statement:
>
> avg1 = mon_avg_obs(m,:,:)
> fmask = where(ismissing(avg1), False, avg1.eq.-2222)
> factors(a*12+m,:,:) = where(fmask, 0., factors(a*12+m,:,:))
>
> print("3. "+num(ismissing(factors(a*12+m,:,:))))
>
> "Lazy evaluation expression does not work for arrays." I wish I had
> seen this sooner.
>
> Dave Allured
> CU/CIRES Climate Diagnostics Center (CDC)
> http://cires.colorado.edu/science/centers/cdc/
> NOAA/ESRL/PSD
> <http://cires.colorado.edu/science/centers/cdc/%0ANOAA/ESRL/PSD>,
> Climate Analysis Branch (CAB)
> *** http://www.esrl.noaa.gov/psd/psd1/ ***
> NOTICE: NEW PSD1/CAB WEBSITE ADDRESS AS OF 2009-OCT-14
> PRODUCT AND DATA CONSUMERS PLEASE TAKE NOTE
>
> Bridget Thrasher wrote:
> > I am doing the following in a loop:
> >
> > mon_avg_obs(m,:,:) = where(mon_avg_obs(m,:,:).eq.0, -2222,
> > mon_avg_obs(m,:,:))
> > factors(a*12+m,:,:) = data_all(a*12+m,:,:)/mon_avg_20c(m,:,:)
> > print("1. "+num(ismissing(factors(a*12+m,:,:))))
> > factors(a*12+m,:,:) = where(.not.ismissing(mon_avg_obs(m,:,:)),
> > dvar(a*12+m,:,:)/mon_avg_obs(m,:,:), factors(a*12+m,:,:))
> > print("2. "+num(ismissing(factors(a*12+m,:,:))))
> > factors(a*12+m,:,:) =
> >
> where(.not.ismissing(mon_avg_obs(m,:,:)).and.mon_avg_obs(m,:,:).eq.-2222,
> > 0., factors(a*12+m,:,:))
> > print("3. "+num(ismissing(factors(a*12+m,:,:))))
> >
> >
> > The output on the screen says:
> > (0) 1. 0
> > (0) 2. 0
> > (0) 3. 11753
> >
> > Why is the last where statement inserting a fill value? I've tried it
> > with and without the .not.ismissing() part, as well as some other
> > things, but all yield the same result.
> >
> > -Bridget
> >
> > --
> > Bridget Thrasher, PhD
> > Postdoctoral Researcher
> > Climate Central
> > www.climatecentral.org <http://www.climatecentral.org>
_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Tue Oct 20 18:27:26 2009