# Strange behavior in algebric computations

From: Mateus Teixeira <mateus.teixeira_at_nyahnyahspammersnyahnyah>
Date: Thu, 1 Oct 2009 11:13:09 -0300

Dear NCL users,

I'm doing some algebric computations that are having strange behavior
depending on way the formula is written. When written this way

su = su + dh * ( u(c,t,k-1,j,i) + u(c,t,k,j,i) ) / 2

I'm getting a missing value (the formula is inside a do loop), as shown below:

(0) sv(before)=0; dh=1271; v(k-1)=0.699997; v(k)=-4.8
(0) sv(after)=-2605.55
(0) sv(before)=-2605.55; dh=2114; v(k-1)=-4.8; v(k)=-10.7
(0) sv(after)=-32767 <= MISSING VALUE

But when the formula is written as

su = su + 0.5 * dh * ( u(c,t,k-1,j,i) + u(c,t,k,j,i) )

I haven't a missingl value, as shown below for the same stop point showed above

(0) sv(before)=0; dh=1271; v(k-1)=0.699997; v(k)=-4.8
(0) sv(after)=-2605.55
(0) sv(before)=-2605.55; dh=2114; v(k-1)=-4.8; v(k)=-10.7
(0) sv(after)=-18989.1 <=CORRECT VALUE

I made these same computations by hand in NCL with both formulae and
get the correct value given by the second way of the formula.

Can anyone imagine what is causing this? I thought that the float
representation of the 1/2 could be the source of the error, but the
difference between missing value and the correct value is not little.

Best regards,

```--
Mateus da Silva Teixeira
Meteorologista
Instituto de Pesquisas Meteorológicas - IPMet