Re: string to short conversion: works on linux but not on a mac, both 64-bit

Hi Gina,

I haven't seen this, but it would help if you could provide some sample code that shows the problem. There may be a slight round-off error between the two systems.

The error is stemming from trying to convert a double to a short, and that double is too large to be represent by a float. You can reproduce this with:

ncl 0> d = 1d20
ncl 1> s = toshort(d)

As a result, "s" will be missing:

ncl 2> print(s)
Variable: s
Type: short
Total Size: 2 bytes
            1 values
Number of Dimensions: 1
Dimensions and sizes: [1]
Coordinates:
Number Of Attributes: 1
  _FillValue : -32767
(0) -32767
ncl 3>

Note that the largest value that a short can be is (2^15)-1.

You can see this with:

ncl 0> d = 2d^15
ncl 1> print(toshort(d))
warning:toshort: there are 1 double larger than SHRT_MAX, which has been flagged missing.
(0) -32767
ncl 2> d = (2d^15)-1
ncl 3> print(toshort(d))
(0) 32767
ncl 4>

--Mary

On Oct 10, 2011, at 9:33 AM, Gina Henderson wrote:

> Hi all,
>
> I have encountered a problem when trying to use "tostring" or "tofloat" in a script on Mac (10.7, 64-bit). When the script gets to this point it hangs and eventually returns the error:
>
> warning:toshort: there are 1048576 double larger than SHRT_MAX, which has been flagged missing.
>
> If I run the same script on a linux system, also 64-bit, it runs fine. Has anyone encountered something similar to this before?
>
> Gina.
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Mon Oct 10 09:43:17 2011