Re: A truncation issue with tointeger()?

I am not sure whether this is related to the same issue, but this is the simplified version of how I further used tmpV and ispan to define the new array and I don't quite get either why I don't get nice rounded values for this rather simple calculation:

        tmp1 = -1.0 + ispan(0,20,1) * 0.1
        print(tmp1)

Variable: tmp1
Type: float
Total Size: 84 bytes
            21 values
Number of Dimensions: 1
Dimensions and sizes: [21]
Coordinates:
-1
-0.9
-0.8
-0.7
-0.6
-0.5
-0.4
-0.3
-0.2
-0.09999996
 0
0.1
0.2
0.3000001
0.4
0.5
0.6
0.7
0.8000001
0.9
 1

Why isn't it -0.1, 0.3 and 0.8?

Emilie

On Jan 6, 2011, at 15:43, James Correia wrote:

> I don't know if this is related or not, but numeric types dont have the decimal, while float and integer do. It looks like tointeger must do rounding, but I am just guessing.
> jimmyc
>
> On Thu, Jan 6, 2011 at 4:27 PM, Emilie Vanvyve <evanvyve@ucar.edu> wrote:
> Hello,
>
> I just bumped into the below problem in my script and I cannot figure out why it happens. I was hoping someone would have an idea...
>
> I have the following 2 float variables:
> tmpV = (/ -1.05, 15.75 /) ; lower and upper bounds of a new array to be later defined with a dvar increment
> dvar = 0.15
>
> I calculate tmp1 as:
> tmp1 = ( tmpV(1) - tmpV(0) ) / dvar
> => tmp1 = float, print(tmp1) shows tmp1 = 112 (no decimal).
>
> If I then calculate tmp2 as:
> tmp2 = tointeger( ( tmpV(1) - tmpV(0) ) / dvar )
> => tmp2 = integer, print(tmp2) shows tmp2 = 111.
>
> Why do I loose a unit here? tmp2 should be equal to 112 since tmp1 doesn't have any decimal, no?
> I need tmp2 to be an integer because it is used with ispan to define that new array I mentioned above.
>
> The on-line documentation specifies that the tointeger() function will truncate the decimals of a float- or double-type variable it is given. However, since I don't appear to have any decimal, why does it look like some truncation is taking place?
>
> Emilie
>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
>
> --
> ------------------------------------------------------------------------------------------
> The views expressed in this email do not necessarily reflect those of NOAA, the National Weather Service, or the University of Oklahoma.
> ------------------------------------------------------------------------------------------
> James Correia Jr.
> OU CIMMS Research Associate
> SPC HWT Liaison
>

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Thu Jan 6 15:59:24 2011