basic math

From: Ruben van Hooidonk <rubski_at_nyahnyahspammersnyahnyah>
Date: Thu, 10 Apr 2008 13:17:13 -0400

Hi,

When I do some basic math in NCL I get unexpected results. I was
wondering if I have made some mistake somewhere or if there is a
solution to prevent this behavior.

The problem is illustrated by this snippet of code:
print(4-4.8)
(0) -0.8000002

This phenomenon is not limited to subtraction:
print(1.2*5.3)
(0) 6.360001

But it is sensitive to the value of the floats:
print(1.2*5.1)
(0) 6.12

This happens while using ncl version 4.2.0.a035 on a PPC mac, version 4.3.1
on an intel linux machine, and while using version 5.0.1 on an intel mac.

Thank you,

Ruben
_______________________________________________
ncl-talk mailing list
ncl-talk_at_ucar.edu
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Thu Apr 10 2008 - 11:17:13 MDT

This archive was generated by hypermail 2.2.0 : Fri Apr 11 2008 - 11:01:26 MDT