Re: Strange behavior in algebric computations

From: Mary Haley <haley_at_nyahnyahspammersnyahnyah>
Date: Thu, 1 Oct 2009 10:01:26 -0600

Hi Mateus,

I was unable to reproduce your problem. Can you try this code and let
me know what you get:

su = 0.
dh = 1271
v1 = 0.699997
v2 = -4.8
su1 = su + dh * (v1 + v2) / 2
su2 = su + 0.5 * dh * (v1 + v2)
print("su1 = " + su1)
print("su2 = " + su2)

su = -2605.55
dh = 2114
v1 = -4.8
v2 = -10.7
su1 = su + dh * (v1 + v2) / 2
su2 = su + 0.5 * dh * (v1 + v2)
print("su1 = " + su1)
print("su2 = " + su2)

My first thought is that this has something to do with the types of
your variables, and the order of which the arithmetic is being done
on them.

Can you do a "printVarSummary" on all your variables and send me the
output so I have the correct types?

--Mary

On Oct 1, 2009, at 8:13 AM, Mateus Teixeira wrote:

> Dear NCL users,
>
> I'm doing some algebric computations that are having strange behavior
> depending on way the formula is written. When written this way
>
> su = su + dh * ( u(c,t,k-1,j,i) + u(c,t,k,j,i) ) / 2
>
> I'm getting a missing value (the formula is inside a do loop), as
> shown below:
>
> (0) sv(before)=0; dh=1271; v(k-1)=0.699997; v(k)=-4.8
> (0) sv(after)=-2605.55
> (0) sv(before)=-2605.55; dh=2114; v(k-1)=-4.8; v(k)=-10.7
> (0) sv(after)=-32767 <= MISSING VALUE
>
> But when the formula is written as
>
> su = su + 0.5 * dh * ( u(c,t,k-1,j,i) + u(c,t,k,j,i) )
>
> I haven't a missingl value, as shown below for the same stop point
> showed above
>
> (0) sv(before)=0; dh=1271; v(k-1)=0.699997; v(k)=-4.8
> (0) sv(after)=-2605.55
> (0) sv(before)=-2605.55; dh=2114; v(k-1)=-4.8; v(k)=-10.7
> (0) sv(after)=-18989.1 <=CORRECT VALUE
>
> I made these same computations by hand in NCL with both formulae and
> get the correct value given by the second way of the formula.
>
> Can anyone imagine what is causing this? I thought that the float
> representation of the 1/2 could be the source of the error, but the
> difference between missing value and the correct value is not little.
>
> Best regards,
>
> --
> Mateus da Silva Teixeira
> Meteorologista
> Instituto de Pesquisas Meteorológicas - IPMet
> Av. Luis Edmundo Carrijo Coube, 14-01 - Cx. Postal 281 - CEP 17033-360
> Bauru - SP - Brasil
> Fone: +55 14 3103-6030 / fax: 3203-3649
>
> Registered Linux User #466740 (http://counter.li.org/)
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Thu Oct 01 2009 - 10:01:26 MDT

This archive was generated by hypermail 2.2.0 : Mon Oct 05 2009 - 13:28:34 MDT