Re: A truncation issue with tointeger()?

From: James Correia <jimmyc42_at_nyahnyahspammersnyahnyah>
Date: Thu Jan 06 2011 - 15:43:08 MST

I don't know if this is related or not, but numeric types dont have the
decimal, while float and integer do. It looks like tointeger must do
rounding, but I am just guessing.
jimmyc

On Thu, Jan 6, 2011 at 4:27 PM, Emilie Vanvyve <evanvyve@ucar.edu> wrote:

> Hello,
>
> I just bumped into the below problem in my script and I cannot figure out
> why it happens. I was hoping someone would have an idea...
>
> I have the following 2 float variables:
> tmpV = (/ -1.05, 15.75 /) ; lower and upper bounds of a new array
> to be later defined with a dvar increment
> dvar = 0.15
>
> I calculate tmp1 as:
> tmp1 = ( tmpV(1) - tmpV(0) ) / dvar
> => tmp1 = float, print(tmp1) shows tmp1 = 112 (no decimal).
>
> If I then calculate tmp2 as:
> tmp2 = tointeger( ( tmpV(1) - tmpV(0) ) / dvar )
> => tmp2 = integer, print(tmp2) shows tmp2 = 111.
>
> Why do I loose a unit here? tmp2 should be equal to 112 since tmp1 doesn't
> have any decimal, no?
> I need tmp2 to be an integer because it is used with ispan to define that
> new array I mentioned above.
>
> The on-line documentation specifies that the tointeger() function will
> truncate the decimals of a float- or double-type variable it is given.
> However, since I don't appear to have any decimal, why does it look like
> some truncation is taking place?
>
> Emilie
>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>

-- 
------------------------------------------------------------------------------------------
The views expressed in this email do not necessarily reflect those of NOAA,
the National Weather Service, or the University of Oklahoma.
------------------------------------------------------------------------------------------
James Correia Jr.
OU CIMMS Research Associate
SPC HWT Liaison

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Thu Jan 6 15:43:16 2011

This archive was generated by hypermail 2.1.8 : Thu Jan 13 2011 - 09:24:21 MST