Re: NCL is getting slower when I call "user defined function" many times

Hi Joowan,

We have been able to duplicate the issue you raise here. The problem is that there are some small objects that are created in great numbers but do not get removed as the do loops progress. They do take up memory but the real cause of the slowdown is that access to the object table slows as the table gets really large. I will try to get to the bottom of this problem as soon as I can.

  -dave

On Apr 22, 2013, at 10:31 AM, Joowan Kim <joowan.k@gmail.com> wrote:

Dear NCL users 

I'm calculating the tropopause for temperature profiles using NCL's "do loop" and a user-defined function. 

The user-defined function works for a single temperature profile, and do loop call the function for each temperature profiles.

There is an exactly same calculation ("just calling of a function I defined")  in the do loop, so each step is expected to take a same time. However the do loop takes longer and longer time as it progresses.

Here is the example 

(elapsed time (time1-time0) for each step keeps increasing) 

[joowan@rossby ttl_joowan]$ ncl main.ncl

 Copyright (C) 1995-2011 - All Rights Reserved

 University Corporation for Atmospheric Research

 NCAR Command Language Version 6.0.0

 The use of this software is governed by a License Agreement.

 See http://www.ncl.ucar.edu/ for more details.

(0) loop i = 1  elapsed time: 4

(0) loop i = 2  elapsed time: 5

(0) loop i = 3  elapsed time: 7

(0) loop i = 4  elapsed time: 9

(0) loop i = 5  elapsed time: 10

(0) loop i = 6  elapsed time: 11

(0) loop i = 7  elapsed time: 13

(0) loop i = 8  elapsed time: 14

(0) loop i = 9  elapsed time: 15

(0) loop i = 10  elapsed time: 17

(0) loop i = 11  elapsed time: 19

(0) loop i = 12  elapsed time: 20

...

...

(0) loop i = 31  elapsed time: 52

(0) loop i = 32  elapsed time: 54

I tested this with the newest version 6.1.2, but it has the same problem.

It is very strange because I never met this problem with NCL before.

One thing I found is memory use keeps increase until unreasonable size. I guess "user defined function" doesn't return memory, so lots of calling it takes bigger memory and longer time. (If I directly do the calculation in the code, not using a user-defined function, that slow down does NOT happen.) But it is just my crude guess...  

It is a serious problem for me, because the do loop goes up to 3000 for model analysis, and the computation time grows exponentially.

Is there any Known issue for this problem or treatment?

Any comment will be appreciated.

Thanks so much

Best!

Just in case, I attached my script and data

Joowan

 

P.S. 

I'm using 

Linux rossby.meteo.mcgill.ca 2.6.34.9-69.fc13.x86_64 #1 SMP Tue May 3 09:23:03 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux

GCC version is "gcc version 4.4.5 20101112 (Red Hat 4.4.5-2) (GCC)"

--
Joowan Kim
Atmospheric and Oceanic Sciences
McGill University
e-mail : joowan.kim@mail.mcgill.ca
                joowan.k@gmail.com

<main.ncl><ttl_joowan.ncl><tprof_ERAI.dat>_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Tue Apr 23 12:28:14 2013