Re: Fwd: Re: Fatal Error Message

You may try the where function:

http://www.ncl.ucar.edu/Document/Functions/Built-in/where.shtml

Prototype

        function where (
                condtnl_expr ,
                true_value ,
                false_value
        )

        return_val [dimsizes(condtnl_expr)]

;Define category first as the dimensions of rr and rc.

 category = 0 ; or a missing value. based on the method of categorizing, and the end, there should be no 0.

 category = where((rr.ge.thres2).and.(rc.ge.thres2), 1, category)
 category = where((rr.lt.thres2).and.(rr.ge.thres1).and.(rc.ge.thres2), 2, category)
 category = where((rr.lt.thres1).and.(rc.ge.thres2), 3, category)
 category = where((rr.ge.thres2).and.(rc.lt.thres2).and.(rc.ge.thres1), 4, category)
 category = where((rr.lt.thres2).and.(rr.ge.thres1).and.(rc.lt.thres2).and.(rc.ge.thres1), 5, category)
 category = where((rr.lt.thres1).and.(rc.lt.thres2).and.(rc.ge.thres1), 6, category)
 category = where((rr.ge.thres2).and.(rc.lt.thres1), 7, category)
 category = where((rr.lt.thres2).and.(rr.ge.thres1).and.(rc.lt.thres1), 8, category)
 category = where((rr.lt.thres1).and.(rc.lt.thres1), 9, category)

At least you can re-org your if-block to:

        if(rr(yy,xx).ge.thres2) then
        else
           if(rr(yy,xx).lt.thres1) then
           else
           end if
        end if

and then insert the rc part if the above, so you have total of 9 categories.
which may also make your code more easy to read.

Wei

On Mar 28, 2014, at 8:24 AM, Dennis Shea <shea@ucar.edu> wrote:

>
>
>
> -------- Original Message --------
> Subject: Re: Fatal Error Message
> Date: Thu, 27 Mar 2014 21:10:15 -0700 (PDT)
> From: Lisa Alexander <lisa_s_a@yahoo.ca>
>
>
>
> I have tried testing a few things this evening and I have nailed it down
> to one particular loop, where I categorize the values comparing two
> 65x65 grids. When I remove it the program runs very quickly. See the
> loop below. Note that 'xx' and 'yy' are the x and y coordinates, 'rr'
> and 'rc' are the two 65x65 grids, and 'category' is the value being
> created for each grid which is later printed to a 65x65 grid .nc file.
> Also note that 'thres1' and 'thres2' are two thresholds by which the
> grids are categorized (I am doing this for a categorized contingency
> table analysis). I am basically comparing each grid point to the same
> one on the other grid and categorizing them accordingly. I am sure this
> must just be an inefficiency thing. If you have any suggestions it would
> be helpful, but please don't worry about it if you are busy.
> I greatly appreciate your help,
> Lisa
>
> do while (yy.lt.ylim)
> ; print(yy)
> do while (xx.lt.xlim)
> ; print(xx)
> if ((rr(yy,xx).ge.thres2).and.(rc(yy,xx).ge.thres2)) then
> ; This refers to r
> category(yy,xx)=1
> ; print(yy+" "+xx)
> else if
> ((rr(yy,xx).lt.thres2).and.(rr(yy,xx).ge.thres1).and.(rc(yy,xx).ge.thres2))
> then
> ; This refers to s
> category(yy,xx)=2
> ; print(yy+" "+xx)
> else if ((rr(yy,xx).lt.thres1).and.(rc(yy,xx).ge.thres2)) then
> ; This refers to t
> category(yy,xx)=3
> ; print(yy+" "+xx)
> else if
> ((rr(yy,xx).ge.thres2).and.(rc(yy,xx).lt.thres2).and.(rc(yy,xx).ge.thres1))
> then
> ; This refers to u
> category(yy,xx)=4
> ; print(yy+" "+xx)
> else if
> ((rr(yy,xx).lt.thres2).and.(rr(yy,xx).ge.thres1).and.(rc(yy,xx).lt.thres2).and.(rc(yy,xx).ge.thres1))
> then
> ; This refers to v
> category(yy,xx)=5
> ; print(yy+" "+xx)
> else if
> ((rr(yy,xx).lt.thres1).and.(rc(yy,xx).lt.thres2).and.(rc(yy,xx).ge.thres1))
> then
> ; This refers to w
> category(yy,xx)=6
> ; print(yy+" "+xx)
> else if ((rr(yy,xx).ge.thres2).and.(rc(yy,xx).lt.thres1)) then
> ; This refers to x
> category(yy,xx)=7
> ; print(yy+" "+xx)
> else if
> ((rr(yy,xx).lt.thres2).and.(rr(yy,xx).ge.thres1).and.(rc(yy,xx).lt.thres1))
> then
> ; This refers to y
> category(yy,xx)=8
> ; print(yy+" "+xx)
> else if ((rr(yy,xx).lt.thres1).and.(rc(yy,xx).lt.thres1)) then
> ; This refers to z
> category(yy,xx)=9
> ; print(yy+" "+xx)
> else
> print("ERROR 2!!!!!!!!!")
> end if
> end if
> end if
> end if
> end if
> end if
> end if
> end if
> end if
> xx=xx+1
> end do
> xx=0
> yy=yy+1
> end do
> --------------------------------------------
> On Thu, 3/27/14, Dennis Shea <shea@ucar.edu> wrote:
>
> Subject: Fwd: Re: Fwd: Re: [ncl-talk] Fatal Error Message
> To: "Lisa Alexander" <lisa_s_a@yahoo.ca>
> Received: Thursday, March 27, 2014, 7:30 PM
>
> Lisa , off-line here
>
> I sent this to a 'core developer.' There have been issues
> with really long strings.
>
> See his comment. Further see his suggestion of using
> 'write_table'
>
> https://www.ncl.ucar.edu/Document/Functions/Built-in/write_table.shtml
>
> ---
> If she is writing a string to a text file every loop
> iteration,
> especially if the string is built up from many small
> components
> concatenated together, that could well be the problem.
> Perhaps using write_table in append mode would allow her to
> avoid
> creating so many unique strings.
> ---
>
> Hope this works.
> D
>
>> -------- Original Message --------
>> Subject: Re: Fatal Error Message
>> Date: Thu, 27 Mar 2014 07:21:23 -0700 (PDT)
>> From: Lisa Alexander <lisa_s_a@yahoo.ca>
>> To: Dennis Shea <shea@ucar.edu>
>> CC: ncl-talk@ucar.edu
> <ncl-talk@ucar.edu>
>>
>> Hi Dennis,
>> Thanks for the input. The array size is not any larger,
> it just loops
>> through finding it many more times.
>> I have 6 GB of memory on my computer. I am not sure
> what my variables
>> require. But it loops through finding the values it
> requires many times
>> before it has a problem. So, there is something
> building up over time,
>> but it does not make sense to me because all variables
> are overwritten
>> on the next loop.
>>
>> One thing that I am doing is appending the results for
> each loop to a
>> .txt file. Could this be causing some problem?
>> I attempted to run the program as you specified below,
> but it does not
>> seem to identify the 'top' command.
>>
>> Thanks very much for your help,
>> Lisa
>> --------------------------------------------
>> On Thu, 3/27/14, Dennis Shea <shea@ucar.edu>
> wrote:
>>
>> Subject: Re: Fatal Error
> Message
>> To: "Lisa Alexander" <lisa_s_a@yahoo.ca>
>> Cc: "ncl-talk@ucar.edu"
> <ncl-talk@ucar.edu>
>> Received: Thursday, March 27, 2014,
> 9:20 AM
>>
>>
>> [1] fatal:NclMalloc Failed:[errno=12]
>>
>> "I have been running a program on
> my
>> computer, but this time
>> the case time length is
> very long ..."
>>
>> I assume this means the
> array size(s) are much
>> larger.
>>
>> You have run out of
> memory.
>>
>> If the primary variable
> is (say) 6GB and you
>> have only 4GB,
>> you will get a
> 'NclMalloc Failed' error. This
>> is regardless
>> if you have deleted all
> other variables.
>>
>> Do you know how much
> memory your computer has
>> *available*?
>> Do you know how much
> memory your main
>> variable(s) need?
>>
>> [2] One crude approach
>>
>> %> ncl
> lisaAlexander.ncl >&! z.out
>> &
>> Then immediately,
>> %> top
>>
>> This will show you the
> memory usage
>> of your job.
>>
>>
>> On 3/26/14, 4:12 PM, Lisa Alexander
> wrote:
>>> Hello,
>>> Please see the problem I have
> been experiencing below
>> (noted in a previous post). I have
> tried to delete all
>> variables once they are not being used
> and this does not
>> work (does not improve the situation
> even slightly).
>>> Would it be possible for someone
> to take a look at the
>> program to see if there are any
> obvious sinks for computer
>> memory?
>>> Thanks for your help,
>>> Lisa
>>>
>>> --- On Wed, 3/26/14, Lisa
> Alexander <lisa_s_a@yahoo.ca>
>> wrote:
>>>
>>>> From: Lisa Alexander <lisa_s_a@yahoo.ca>
>>>> Subject: Fatal
> Error Message
>>>> To: ncl-talk@ucar.edu
>>>> Received: Wednesday, March
> 26, 2014, 2:56 PM
>>>> Hello,
>>>> I have been running a program
> on my computer, but
>> this time
>>>> the case time length is very
> long. When I run it,
>> it stops
>>>> part way through and gives me
> this message:
>>>>
>>>> fatal:NclMalloc
> Failed:[errno=12]
>>>> Segmentation fault (core
> dumped)
>>>>
>>>> I must say, whenever I run
> the program (this time
>> and other
>>>> times), the program runs very
> quickly to begin
>> with, but
>>>> slows down over time.
>>>>
>>>> Can somebody let me know what
> the error message
>> means?
>>>> Thanks,
>>>> Lisa
>>>>
> _______________________________________________
>>>> ncl-talk mailing list
>>>> List instructions, subscriber
> options,
>> unsubscribe:
>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>
>>>
> _______________________________________________
>>> ncl-talk mailing list
>>> List instructions, subscriber
> options, unsubscribe:
>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>
>>
>>
>>
>>
>
>
>
>
>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Fri Mar 28 09:15:03 2014