Re: NCL is getting slower when I call "user defined function" many times

From: Joowan Kim <joowan.k_at_nyahnyahspammersnyahnyah>
Date: Tue Apr 23 2013 - 19:49:57 MDT

Dear David

Thanks for your effort and kind explanation.
I appreciate it!

Regards,

Joowan

On 23 April 2013 14:15, David Brown <dbrown@ucar.edu> wrote:

> Hi Joowan,
>
> We have been able to duplicate the issue you raise here. The problem is
> that there are some small objects that are created in great numbers but do
> not get removed as the do loops progress. They do take up memory but the
> real cause of the slowdown is that access to the object table slows as the
> table gets really large. I will try to get to the bottom of this problem as
> soon as I can.
> -dave
>
> On Apr 22, 2013, at 10:31 AM, Joowan Kim <joowan.k@gmail.com> wrote:
>
> Dear NCL users
>
> I'm calculating the tropopause for temperature profiles using NCL's "do
> loop" and a user-defined function.
> The user-defined function works for a single temperature profile, and do
> loop call the function for each temperature profiles.
> There is an exactly same calculation ("just calling of a function I
> defined") in the do loop, so each step is expected to take a same time.
> However the do loop takes longer and longer time as it progresses.
>
> Here is the example
> (elapsed time (time1-time0) for each step keeps increasing)
> [joowan@rossby ttl_joowan]$ ncl *main.ncl*
> Copyright (C) 1995-2011 - All Rights Reserved
> University Corporation for Atmospheric Research
> NCAR Command Language Version 6.0.0
> The use of this software is governed by a License Agreement.
> See http://www.ncl.ucar.edu/ for more details.
> (0) loop i = 1 elapsed time: 4
> (0) loop i = 2 elapsed time: 5
> (0) loop i = 3 elapsed time: 7
> (0) loop i = 4 elapsed time: 9
> (0) loop i = 5 elapsed time: 10
> (0) loop i = 6 elapsed time: 11
> (0) loop i = 7 elapsed time: 13
> (0) loop i = 8 elapsed time: 14
> (0) loop i = 9 elapsed time: 15
> (0) loop i = 10 elapsed time: 17
> (0) loop i = 11 elapsed time: 19
> (0) loop i = 12 elapsed time: 20
> ...
> ...
> (0) loop i = 31 elapsed time: 52
> (0) loop i = 32 elapsed time: 54
>
> I tested this with the newest version 6.1.2, but it has the same problem.
> It is very strange because I never met this problem with NCL before.
> One thing I found is memory use keeps increase until unreasonable size. I
> guess "user defined function" doesn't return memory, so lots of calling it
> takes bigger memory and longer time. (If I directly do the calculation in
> the code, not using a user-defined function, that slow down does NOT
> happen.) But it is just my crude guess...
> It is a serious problem for me, because the do loop goes up to 3000 for
> model analysis, and the computation time grows exponentially.
> Is there any Known issue for this problem or treatment?
> Any comment will be appreciated.
>
> Thanks so much
>
> Best!
> Just in case, I attached my script and data
>
>
> Joowan
>
> P.S.
> I'm using
> Linux rossby.meteo.mcgill.ca 2.6.34.9-69.fc13.x86_64 #1 SMP Tue May 3
> 09:23:03 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
> GCC version is "gcc version 4.4.5 20101112 (Red Hat 4.4.5-2) (GCC)"
>
>
> --
> Joowan Kim
> Atmospheric and Oceanic Sciences
> McGill University
> e-mail : joowan.kim@mail.mcgill.ca
> joowan.k@gmail.com
> <main.ncl><ttl_joowan.ncl><tprof_ERAI.dat>
> _______________________________________________
>
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
>

-- 
Joowan Kim
Atmospheric and Oceanic Sciences
McGill University
e-mail : joowan.kim@mail.mcgill.ca
                joowan.k@gmail.com

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Tue Apr 23 19:50:09 2013

This archive was generated by hypermail 2.1.8 : Fri Apr 26 2013 - 17:10:26 MDT