Re: A truncation issue with tointeger()?

Hi James,
I am not sure I understand what you mean. My 112 value is from a float variable. I would have expected it to display as 11.99999999999999 or something if the calculation wasn't leading to an exact 112 (which it should, but who knows) and I would then understand why tointeger() gives me 111 (the .99999999 part being then chopped).

Emilie

On Jan 6, 2011, at 15:43, James Correia wrote:

> I don't know if this is related or not, but numeric types dont have the decimal, while float and integer do. It looks like tointeger must do rounding, but I am just guessing.
> jimmyc
>
> On Thu, Jan 6, 2011 at 4:27 PM, Emilie Vanvyve <evanvyve@ucar.edu> wrote:
> Hello,
>
> I just bumped into the below problem in my script and I cannot figure out why it happens. I was hoping someone would have an idea...
>
> I have the following 2 float variables:
> tmpV = (/ -1.05, 15.75 /) ; lower and upper bounds of a new array to be later defined with a dvar increment
> dvar = 0.15
>
> I calculate tmp1 as:
> tmp1 = ( tmpV(1) - tmpV(0) ) / dvar
> => tmp1 = float, print(tmp1) shows tmp1 = 112 (no decimal).
>
> If I then calculate tmp2 as:
> tmp2 = tointeger( ( tmpV(1) - tmpV(0) ) / dvar )
> => tmp2 = integer, print(tmp2) shows tmp2 = 111.
>
> Why do I loose a unit here? tmp2 should be equal to 112 since tmp1 doesn't have any decimal, no?
> I need tmp2 to be an integer because it is used with ispan to define that new array I mentioned above.
>
> The on-line documentation specifies that the tointeger() function will truncate the decimals of a float- or double-type variable it is given. However, since I don't appear to have any decimal, why does it look like some truncation is taking place?
>
> Emilie
>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
>
> --
> ------------------------------------------------------------------------------------------
> The views expressed in this email do not necessarily reflect those of NOAA, the National Weather Service, or the University of Oklahoma.
> ------------------------------------------------------------------------------------------
> James Correia Jr.
> OU CIMMS Research Associate
> SPC HWT Liaison
>

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Thu Jan 6 15:46:55 2011