Problem with averaging integers?

From: Charles Bardeen <bardeenc_at_nyahnyahspammersnyahnyah>
Date: Sun Oct 16 2011 - 15:52:13 MDT

The following code yields an average that is 1 from the true average …

 Copyright (C) 1995-2010 - All Rights Reserved
 University Corporation for Atmospheric Research
 NCAR Command Language Version 5.2.1
 The use of this software is governed by a License Agreement.
 See http://www.ncl.ucar.edu/ for more details.
ncl 0> tmp = new((/ 60 /), integer)
ncl 1> tmp(:) = 20060109
ncl 2> print(avg(tmp))
(0) 2.006011e+07
ncl 3> print(doubletointeger(avg(tmp)))
(0) 20060108
ncl 4>

Any idea why NCL is making this mistake. Is it an overflow or rounding problem? If I use smaller numbers, then it works correctly …

ncl 8> tmp(:) = 2006
ncl 9> print(avg(tmp))
(0) 2006
ncl 10> print(doubletointeger(avg(tmp)))
(0) 2006
ncl 11>

Thanks,

Chuck

--------------------------------------------------------------
Charles Bardeen
National Center for Atmospheric Research
P.O. Box 3000, Boulder, CO 80307-3000
Phone: (303) 497-1752, Fax: (303) 497-1400
bardeenc@ucar.edu

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Sun Oct 16 15:52:19 2011

This archive was generated by hypermail 2.1.8 : Tue Oct 18 2011 - 09:42:15 MDT