strange arithmetic error

From: Walter Hannah <whannah_at_nyahnyahspammersnyahnyah>
Date: Tue Nov 20 2012 - 11:31:10 MST

I've come across some strange arithmetic in NCL (6.1 beta). Here's a
copy and paste from a terminal window to demonstrate the problem:

    Copyright (C) 1995-2012 - All Rights Reserved
      University Corporation for Atmospheric Research
      NCAR Command Language Version 6.1.0-beta
      The use of this software is governed by a License Agreement.
      See http://www.ncl.ucar.edu/ for more details.
    ncl 0> print( 20110001-20110000 )
    (0) 1
    ncl 1> print( 20110001.-20110000. )
    (0) 0

The second answer should obviously be 1.0, but its not!!!! So when using
floating point numbers there seems to besome weird round off problem or
something. Am I missing something here? I tested some other things and
got other strange results like this one:

    ncl 5> print( 2011.0001-2011. )
    (0) 0.0001220703

Should I roll back a version and see if its still happens?

Walter Hannah

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Tue Nov 20 11:31:19 2012

This archive was generated by hypermail 2.1.8 : Wed Nov 21 2012 - 11:16:05 MST