Re: Lazy expression evaluation

From: David Brown <dbrown_at_nyahnyahspammersnyahnyah>
Date: Wed, 17 Jun 2009 14:26:59 -0600

Dave A.,

My view is that the 'ismissing' function is an explicit exception to
the rule you quote (otherwise it would return a missing value itself
instead of True) whose main purpose is to allow you to work around
the rule when necessary. I think the 'check1' statement below behaves
properly because by using 'ismissing' and lazy evaluation the right
hand side of the '.and.' expression never gets evaluated and
therefore does not figure into the result of the expression as a whole.

I agree with your original premise that 'check2' should work the same
way, and I now think it is a bug that it does not.
By the way I found another reference to lazy evaluation in the NCL
glossary. It is discussed without reference to 'if' statements
although it does not talk about array logical expressions either:

lazy evaluation
NCL: The process whereby relational expressions are assigned a value
as soon as it is possible to do so, without necessarily evaluating
all of the components in the expression. For example, the expression
(1 .lt. 3) .or. (2 .lt. 1) can be assigned the value True immediately
after evaluating (1 .lt. 3) without having to evaluate (2 .lt. 1).


On Jun 16, 2009, at 7:04 PM, Dave Allured wrote:

> Dave B,
> On the other hand, the missing value rule for general expressions,
> logical and others, is elegantly stated on the Expressions page:
> "When any NCL expression is being evaluated, NCL ignores
> elements that are equal to the value of the "_FillValue"
> attribute for each variable. When a missing value is
> ignored, the result of the expression will contain a
> missing value at the corresponding array index."
> There are other contexts in which I would really not want to have
> complications added to this rule. With full knowledge I would
> probably vote for keeping logical expressions in conformance with
> this simple rule.
> This means that (a) a logical expression within an "if" statement
> must explicitly be an exception to this rule, with some
> dimensionality and nesting considerations; and (b) the assignment
> statement for check1 below is not in compliance, and might be a bug
> (which you already said).
> This is reminding me of a saying, "Be careful what you ask for!"
> --Dave
> Dave Allured wrote:
>> Dave B,
>> Thanks for looking at this. I think that lazy expression evaluation
>> for array expressions would be beneficial.
>> It seems to me that that lazy evaluation was never implemented in
>> NCL just for "if" statements. Lazy evaluation also works in general
>> scalar expressions, just not in array expressions. In this example
>> for current NCL versions, check1 is a scalar expression assignment
>> which could not have the indicated result without lazy evaluation:
>> a = (/ 1,2,3 /)
>> a@_FillValue = 2
>> check1 = (.not.ismissing(a(1)) .and. (a(1) .gt. 0))
>> check2 = (.not.ismissing(a) .and. (a .gt. 0))
>> print (check1)
>> print (check2(1))
>> (0) False
>> (0) Missing
>> Also I see in the same documentation under "if statements", there is
>> an almost explicit reference to using the ismissing function in
>> array mode with lazy expression evaluation. "The function ismissing
>> returns an array ..." "Combined with lazy conditional expression
>> evaluation..." There are other suggestive statements in the same
>> section. This seems like simply an incomplete implementation of
>> lazy evaluation.
>> Meanwhile, here is a workaround that I have started to use. This
>> needs only one extra line, and it does not depend on lazy
>> evaluation. x is an array; the output vmask is also an array of the
>> same dimensionality:
>> vmask = (x .gt. -130. .and. x .lt. 130.)
>> vmask = where (ismissing (vmask), False, vmask)
>> This is surely not as efficient as true lazy evaluation in a single
>> line, but it will do for now.
>> --Dave
>> David Brown wrote:
>>> Hi Dave,
>>> This is a very interesting observation. In the NCL reference
>>> manual lazy
>>> expression evaluation is only documented in the context of 'if'
>>> statements, which require a scalar logical expression. The
>>> documentation
>>> for '.and.' and '.or.' says only that the operands must be
>>> logical, but
>>> does not mention lazy evaluation. Apparently lazy evaluation was
>>> implemented specifically for 'if' statement evaluation but was never
>>> generalized to work for array logical expressions.
>>> This leads to the inconsistency that you have pointed out here.
>>> The code
>>> could easily be updated to use lazy evaluation for .and. and .or.
>>> in the
>>> context of array logical expressions. I am not totally confident
>>> that
>>> there might not be some backwards-compatibility issue, but it
>>> does seem
>>> like a bug of sorts, so my inclination is to go ahead and make the
>>> change, noting that the behavior should be clearly documented. The
>>> development team will discuss.
>>> -dave
>>> On Jun 12, 2009, at 7:22 PM, Dave Allured wrote:
>>>> NCL team,
>>>> Is lazy expression evaluation supposed to work for array
>>>> expressions?
>>>> See attached script. mask1 is scalar and shows the expected
>>>> result.
>>>> This is basically the example in the NCL manual under "If
>>>> statements",
>>>> with assignment rather than if statement.
>>>> NclStatements.shtml
>>>> For mask2 I expect True, False, True, but NCL returns True,
>>>> Missing,
>>>> True. This creates problems for subsequent usage of the mask.
>>>> I checked this with NCL versions 5.0.1 (pre-release, ca. May
>>>> 2008) and
>>>> 5.1.1 (pre-release). The problem was the same in both.
>>>> uname -a
>>>> Darwin 9.7.0 Darwin Kernel Version
>>>> 9.7.0: Tue
>>>> Mar 31 22:54:29 PDT 2009; root:xnu-1228.12.14~1/RELEASE_PPC Power
>>>> Macintosh powerpc PowerMac7,3 Darwin
>>>> Please advise. Thank you for taking a look.
>>>> Dave Allured
>>>> CU/CIRES Climate Diagnostics Center (CDC)
>>>> NOAA/ESRL/PSD, Climate Analysis Branch (CAB)
>>>> ; Test program for lazy expression evaluation.
>>>> ; 2009-jun-13 By Dave Allured, NOAA/PSD/CU/CIRES/CDC.
>>>> begin
>>>> a = (/ 1,2,3 /)
>>>> a@_FillValue = 2
>>>> i=1
>>>> mask1 = (.not.ismissing(a(i)) .and. (a(i) .gt. 0))
>>>> print (mask1)
>>>> mask2 = (.not.ismissing(a) .and. (a .gt. 0))
>>>> print (mask2)
>>>> end
>>>> _______________________________________________
>>>> ncl-talk mailing list
>>>> List instructions, subscriber options, unsubscribe:
>> _______________________________________________
>> ncl-talk mailing list
>> List instructions, subscriber options, unsubscribe:
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:

ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
Received on Wed Jun 17 2009 - 14:26:59 MDT

This archive was generated by hypermail 2.2.0 : Fri Jun 19 2009 - 13:23:25 MDT