Re: NCL is getting slower when I call "user defined function" many times

Hi Joowan,

FYI, I have fixed this problem and your script can now run to completion with essentially no increase in time for iterations of the loop from beginning to end. For the record, it has nothing
to do with whether a function is user-defined or not. The code was losing track of small objects holding scalar values in 2 situations: 1) when taking the short-circuit path for .and. and .or. logical expressions, and 2) when creating the _FillValue attribute for certain local variables.

Let us know if you would like to get an updated binary.
 -dave

On Apr 23, 2013, at 7:49 PM, Joowan Kim <joowan.k@gmail.com> wrote:

> Dear David
>
> Thanks for your effort and kind explanation.
> I appreciate it!
>
> Regards,
>
> Joowan
>
>
>
> On 23 April 2013 14:15, David Brown <dbrown@ucar.edu> wrote:
> Hi Joowan,
>
> We have been able to duplicate the issue you raise here. The problem is that there are some small objects that are created in great numbers but do not get removed as the do loops progress. They do take up memory but the real cause of the slowdown is that access to the object table slows as the table gets really large. I will try to get to the bottom of this problem as soon as I can.
> -dave
>
> On Apr 22, 2013, at 10:31 AM, Joowan Kim <joowan.k@gmail.com> wrote:
>
>> Dear NCL users
>>
>> I'm calculating the tropopause for temperature profiles using NCL's "do loop" and a user-defined function.
>> The user-defined function works for a single temperature profile, and do loop call the function for each temperature profiles.
>> There is an exactly same calculation ("just calling of a function I defined") in the do loop, so each step is expected to take a same time. However the do loop takes longer and longer time as it progresses.
>>
>> Here is the example
>> (elapsed time (time1-time0) for each step keeps increasing)
>> [joowan@rossby ttl_joowan]$ ncl main.ncl
>> Copyright (C) 1995-2011 - All Rights Reserved
>> University Corporation for Atmospheric Research
>> NCAR Command Language Version 6.0.0
>> The use of this software is governed by a License Agreement.
>> See http://www.ncl.ucar.edu/ for more details.
>> (0) loop i = 1 elapsed time: 4
>> (0) loop i = 2 elapsed time: 5
>> (0) loop i = 3 elapsed time: 7
>> (0) loop i = 4 elapsed time: 9
>> (0) loop i = 5 elapsed time: 10
>> (0) loop i = 6 elapsed time: 11
>> (0) loop i = 7 elapsed time: 13
>> (0) loop i = 8 elapsed time: 14
>> (0) loop i = 9 elapsed time: 15
>> (0) loop i = 10 elapsed time: 17
>> (0) loop i = 11 elapsed time: 19
>> (0) loop i = 12 elapsed time: 20
>> ...
>> ...
>> (0) loop i = 31 elapsed time: 52
>> (0) loop i = 32 elapsed time: 54
>>
>> I tested this with the newest version 6.1.2, but it has the same problem.
>> It is very strange because I never met this problem with NCL before.
>> One thing I found is memory use keeps increase until unreasonable size. I guess "user defined function" doesn't return memory, so lots of calling it takes bigger memory and longer time. (If I directly do the calculation in the code, not using a user-defined function, that slow down does NOT happen.) But it is just my crude guess...
>> It is a serious problem for me, because the do loop goes up to 3000 for model analysis, and the computation time grows exponentially.
>> Is there any Known issue for this problem or treatment?
>> Any comment will be appreciated.
>>
>> Thanks so much
>>
>> Best!
>> Just in case, I attached my script and data
>>
>>
>> Joowan
>>
>> P.S.
>> I'm using
>> Linux rossby.meteo.mcgill.ca 2.6.34.9-69.fc13.x86_64 #1 SMP Tue May 3 09:23:03 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
>> GCC version is "gcc version 4.4.5 20101112 (Red Hat 4.4.5-2) (GCC)"
>>
>>
>> --
>> Joowan Kim
>> Atmospheric and Oceanic Sciences
>> McGill University
>> e-mail : joowan.kim@mail.mcgill.ca
>> joowan.k@gmail.com
>> <main.ncl><ttl_joowan.ncl><tprof_ERAI.dat>_______________________________________________
>>
>> ncl-talk mailing list
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
>
>
> --
> Joowan Kim
> Atmospheric and Oceanic Sciences
> McGill University
> e-mail : joowan.kim@mail.mcgill.ca
> joowan.k@gmail.com
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Wed Apr 24 15:16:45 2013