lost precision with string2float?

From: À×öª <leiting2002_at_nyahnyahspammersnyahnyah>
Date: Wed Jul 31 2013 - 01:51:33 MDT


Dear NCLers,
The lost precision in function of stringtofloat is very confusing.
A snapshot of my script is :
¦ä¦ä¦ä
£®£®£®£®£®
datafile(0)="/pan2/projects/hybda/tlei/hbrid-gsi/experiments/dr-processing/dr-obsfit/dr-andrew/dr-new-test/t126-ens3dvar-factq0.005-tune1_temp_24_tl3wk_date_2011072300.txtout"
str_data = asciiread(datafile(0),-1,"string")
print("------------------")
print(str_data)
print("------------------")

print("thinkyy1 "+str_get_field(str_data(2:11),10 ," "))
datarmstr(0,:) = stringtofloat(str_get_field(str_data(2:11),10 ," "))
print("output "+ datarmstr(0,:))
exit
£®£®£®£®£®£®£®

£Ö£Ö

the output was :
^^
(0) ------------------


Variable: str_data
Type: string
Total Size: 184 bytes
            23 values
Number of Dimensions: 1
Dimensions and sizes: [23]
Coordinates:
Number Of Attributes: 1
  _FillValue : missing
(0) t126-ens3dvar-factq0.005-tune1
(1) 24
(2) Level: 950 GL: 8.24180969913 NH: 9.03959532222 SH: 4.20871527148 TR: 2.26410202121
(3) Level: 850 GL: 3.40322092967 NH: 3.62214877696 SH: 2.20873685424 TR: 1.25765875807
(4) Level: 750 GL: 1.64514061274 NH: 1.62367548789 SH: 1.89728488578 TR: 1.76729637694
(5) Level: 650 GL: 1.36298927422 NH: 1.33075976043 SH: 2.75972659747 TR: 1.00703519724
(6) Level: 550 GL: 1.03182087585 NH: 1.00696025671 SH: 1.56631219346 TR: 1.1095806198
(7) Level: 450 GL: 1.32992905798 NH: 1.36271631568 SH: 1.18470011083 TR: 1.01202546831
(8) Level: 350 GL: 1.21252777266 NH: 1.22452270919 SH: 1.4303300868 TR: 0.812163178701
(9) Level: 250 GL: 1.70174899943 NH: 1.72542880914 SH: 1.89613201475 TR: 1.00688183094
(10) Level: 150 GL: 2.54344055051 NH: 2.64716957083 SH: 2.15570650502 TR: 1.92993841914
(11) Level: 50 GL: 3.73654217549 NH: 3.34410145973 SH: 3.07788348411 TR: 5.7543725482
(12) tmpnumLevel: 950 GL: 16381.0 NH: 14216.0 SH: 823.0 TR: 1342.0
(13) tmpnumLevel: 850 GL: 7427.0 NH: 6607.0 SH: 329.0 TR: 491.0
(14) tmpnumLevel: 750 GL: 6001.0 NH: 5334.0 SH: 254.0 TR: 413.0
(15) tmpnumLevel: 650 GL: 8412.0 NH: 7609.0 SH: 303.0 TR: 500.0
(16) tmpnumLevel: 550 GL: 7811.0 NH: 7067.0 SH: 258.0 TR: 486.0
(17) tmpnumLevel: 450 GL: 8011.0 NH: 7102.0 SH: 325.0 TR: 584.0
(18) tmpnumLevel: 350 GL: 7837.0 NH: 6834.0 SH: 517.0 TR: 486.0
(19) tmpnumLevel: 250 GL: 28293.0 NH: 26201.0 SH: 937.0 TR: 1155.0
(20) tmpnumLevel: 150 GL: 6086.0 NH: 5067.0 SH: 441.0 TR: 578.0
(21) tmpnumLevel: 50 GL: 8215.0 NH: 6109.0 SH: 692.0 TR: 1414.0
(22) totnum tot: 104474.0 NH: 92146.0 SH: 4879.0 TR: 7449.0
(0) ------------------

(0) ------------------
(0) thinkyy1 2.26410202121
(1) thinkyy1 1.25765875807
(2) thinkyy1 1.76729637694
(3) thinkyy1 1.00703519724
(4) thinkyy1 1.1095806198
(5) thinkyy1 1.01202546831
(6) thinkyy1 0.812163178701
(7) thinkyy1 1.00688183094
(8) thinkyy1 1.92993841914
(9) thinkyy1 5.7543725482
(0) output 2.2641
(1) output 1.25766
(2) output 1.7673
(3) output 1.00704
(4) output 1.10958
(5) output 1.01203
(6) output 0.812163
(7) output 1.00688
(8) output 1.92994
(9) output 5.75437

VV

It seems,say, the string 2.26410202121 was converted to 2.2641 with quite a few numbers lost.
I tried direct converting the string, namely use "stringtofloat(2.2641020212), the converted result would be exactly the same. Why would such lost numbers happen in the above example?
Your help is appreciated.
Best,
Ting


                                               

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Wed Jul 31 01:51:49 2013

This archive was generated by hypermail 2.1.8 : Thu Aug 01 2013 - 15:55:04 MDT