Re: Speed difference

From: Adrien Mauss <mauss_at_nyahnyahspammersnyahnyah>
Date: Fri, 23 Feb 2007 19:09:12 -0600

Hello again !

I want to thanks Adam who gives me this better way to proceed but it seems that my computer was supporting my incorrect
statement. I responded Adam quite fast to say it was OK while I was using the good file which was already working...

But finally I found where was the loss of speed. Someone from this list told me the Fill_Value were different (tmp2@_FillValue
= 1e+20, intmp@_FillValue = -999). If I delete the Fill_Value before the loop or if I change directly
intmp@_FillValue=tmp@_FillValue, I find my speed again. And the difference is very impressive ! So why was it ok with the
other file ? I think this is due to the missing of a FillValue. There was just a missingValue and I guess the treatment is
different.

So thanks Verica for the remark ! And Adam and the NCL team for their help !

I'll try to be more careful next time !

Adrien

----- Original Message -----
From: Adam Phillips <asphilli_at_cgd.ucar.edu>
Date: Friday, February 23, 2007 4:53 pm
Subject: Re: Speed difference
To: Adrien Mauss <mauss_at_wisc.edu>
Cc: ncl-talk_at_ucar.edu

> To update everyone, this was solved offline. Adrien wished to map 3D
> (time,lat,lon) data onto a 2D (coordinate,time) array.. The easiest
> way
> to handle this was to use coordinate subscripting:
>
> begin
> latlon = asciiread("gridout.cru.data",(/67420,2/),"float")
> lat = latlon(:,1)
> lon = latlon(:,0)
>
> a = addfile("interp_tas.nc","r")
> tmp1 = a->sfct(0:1199,:,:) ; I've just take the 100 first years
>
> intmp=new((/67420,1200/),float)
> do ipt=0,5 ;67420-1
> printVarSummary(tmp1(:,{lat(ipt)},{lon(ipt)}))
> print("lat(ipt)= "+lat(ipt)+", lon(ipt)= "+lon(ipt))
> intmp(ipt,:) = (/ tmp1(:,{lat(ipt)},{lon(ipt)}) /)
> end do
> end
>
> Adrien Mauss wrote:
> > Hello everyone !
> >
> > I have got some strange things about a part of a program. I need to
> pick terrestrial data from a global grid (720,360). For a first file
> which contains some 102 years,
> > this part is very fast (1-2 min) whereas when I use an other file of
> 130 years, it takes more than 2-3 hours. So I've tried each one with
> 100 years, the difference of
> > speed is the same. I've looked for the origins and finally, I've
> found the part which was so long. Here the minimum script which shows
> the problem :
> >
> >
> > "begin
> >
> > latlon =
> asciiread("/Users/adrien/data/PARAMETERS/gridout.cru.data",(/67420,2/),"float")
> ; loading the grid of terrestrial points
> > lat = latlon(:,1)
> > lon = latlon(:,0)
> > print("grid data read")
> >
> > print("TMP")
> > a =
> addfile("/Users/adrien/data/INPUT/cru_ts_2_10.1901-2002.tmp.nc","r") ;
> loading the temperature for the global grid
> > tmp1 = a->tmp(0:1199,:,:) ; I've just take the 100 first years
> >
> > latit = a->lat ; latitude and longitude
> > longit = a->lon
> >
> > tmp2=(/tmp1(:,ind(latit.eq.lat(0)),ind(longit.eq.lon(0)))/)
> > ;tmp2=fspan(0.,1199.,1200)
> > printVarSummary(tmp2)
> >
> > intmp=new((/67420,1200/),float)
> > do ipt=0,67420-1
> >
> ;intmp(ipt,:)=tmp1(:,ind(latit.eq.lat(ipt)),ind(longit.eq.lon(ipt)))
> ; the initial command
> > ;tmp2=tmp1(:,ind(latit.eq.lat(ipt)),ind(longit.eq.lon(ipt)))
> ; first part : picking the temperature of the point ; alone this
> command is very fast
> > intmp(ipt,:)=(/tmp2/)
> ; second part : WHERE I LOOSE ALL THE SPEED
> > print("tmp : "+ipt)
> ; in order to see the speed
> > end do
> >
> > end"
> >
> > Variable: tmp2
> > Type: float
> > Total Size: 4800 bytes
> > 1200 values
> > Number of Dimensions: 1
> > Dimensions and sizes: [1200]
> > Coordinates:
> > Number Of Attributes: 1
> > _FillValue : 1e+20
> >
> >
> > So the problem appears when I assign all the values to one point
> (lat,lon) even if I use a constant vector from the first point. But
> when I'm using a other vector
> > (fspan(1.,1199.,1200)) of same size and same type (float) as it's
> shown by printVarSummary(), it's becoming faster (>50 times faster). I
> was thinking every float
> > vector would have a same "behavior". I can't understand why the
> treatment is not the same one. Perhaps, I did something wrong. If
> somebody has a gimmick to
> > make it faster. I can wait but I will be using it 4 times in the
> program, and I should use the program more than 50 times, so it would
> be better to have something
> > fast...
> >
> > Thanks
> >
> > Adrien Mauss
> > mauss_at_wisc.edu
> >
> > --------------------------
> >
> > _______________________________________________
> > ncl-talk mailing list
> > ncl-talk_at_ucar.edu
> >
_______________________________________________
ncl-talk mailing list
ncl-talk_at_ucar.edu
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Fri Feb 23 2007 - 18:09:12 MST

This archive was generated by hypermail 2.2.0 : Mon Mar 05 2007 - 09:27:29 MST