Fwd: Re: Fatal Error Message

-------- Original Message --------
Subject: Re: Fatal Error Message
Date: Thu, 27 Mar 2014 21:10:15 -0700 (PDT)
From: Lisa Alexander <lisa_s_a@yahoo.ca>

I have tried testing a few things this evening and I have nailed it down
to one particular loop, where I categorize the values comparing two
65x65 grids. When I remove it the program runs very quickly. See the
loop below. Note that 'xx' and 'yy' are the x and y coordinates, 'rr'
and 'rc' are the two 65x65 grids, and 'category' is the value being
created for each grid which is later printed to a 65x65 grid .nc file.
Also note that 'thres1' and 'thres2' are two thresholds by which the
grids are categorized (I am doing this for a categorized contingency
table analysis). I am basically comparing each grid point to the same
one on the other grid and categorizing them accordingly. I am sure this
must just be an inefficiency thing. If you have any suggestions it would
be helpful, but please don't worry about it if you are busy.
I greatly appreciate your help,
Lisa

   do while (yy.lt.ylim)
; print(yy)
        do while (xx.lt.xlim)
; print(xx)
                if ((rr(yy,xx).ge.thres2).and.(rc(yy,xx).ge.thres2)) then
; This refers to r
                        category(yy,xx)=1
; print(yy+" "+xx)
                else if
((rr(yy,xx).lt.thres2).and.(rr(yy,xx).ge.thres1).and.(rc(yy,xx).ge.thres2))
then
; This refers to s
                        category(yy,xx)=2
; print(yy+" "+xx)
         else if ((rr(yy,xx).lt.thres1).and.(rc(yy,xx).ge.thres2)) then
; This refers to t
                    category(yy,xx)=3
; print(yy+" "+xx)
                else if
((rr(yy,xx).ge.thres2).and.(rc(yy,xx).lt.thres2).and.(rc(yy,xx).ge.thres1))
then
; This refers to u
                        category(yy,xx)=4
; print(yy+" "+xx)
         else if
((rr(yy,xx).lt.thres2).and.(rr(yy,xx).ge.thres1).and.(rc(yy,xx).lt.thres2).and.(rc(yy,xx).ge.thres1))
then
; This refers to v
                    category(yy,xx)=5
; print(yy+" "+xx)
                else if
((rr(yy,xx).lt.thres1).and.(rc(yy,xx).lt.thres2).and.(rc(yy,xx).ge.thres1))
then
; This refers to w
                    category(yy,xx)=6
; print(yy+" "+xx)
         else if ((rr(yy,xx).ge.thres2).and.(rc(yy,xx).lt.thres1)) then
; This refers to x
                    category(yy,xx)=7
; print(yy+" "+xx)
                else if
((rr(yy,xx).lt.thres2).and.(rr(yy,xx).ge.thres1).and.(rc(yy,xx).lt.thres1))
then
; This refers to y
                    category(yy,xx)=8
; print(yy+" "+xx)
                else if ((rr(yy,xx).lt.thres1).and.(rc(yy,xx).lt.thres1)) then
; This refers to z
                        category(yy,xx)=9
; print(yy+" "+xx)
                else
                        print("ERROR 2!!!!!!!!!")
                end if
                end if
                end if
                end if
                end if
                end if
                end if
                end if
                end if
            xx=xx+1
     end do
     xx=0
     yy=yy+1
   end do
--------------------------------------------
On Thu, 3/27/14, Dennis Shea <shea@ucar.edu> wrote:

  Subject: Fwd: Re: Fwd: Re: [ncl-talk] Fatal Error Message
  To: "Lisa Alexander" <lisa_s_a@yahoo.ca>
  Received: Thursday, March 27, 2014, 7:30 PM

  Lisa , off-line here

  I sent this to a 'core developer.' There have been issues
  with really long strings.

  See his comment. Further see his suggestion of using
  'write_table'

  https://www.ncl.ucar.edu/Document/Functions/Built-in/write_table.shtml

  ---
  If she is writing a string to a text file every loop
  iteration,
  especially if the string is built up from many small
  components
  concatenated together, that could well be the problem.
  Perhaps using write_table in append mode would allow her to
  avoid
  creating so many unique strings.
  ---

  Hope this works.
  D

> -------- Original Message --------
> Subject: Re: Fatal Error Message
> Date: Thu, 27 Mar 2014 07:21:23 -0700 (PDT)
> From: Lisa Alexander <lisa_s_a@yahoo.ca>
> To: Dennis Shea <shea@ucar.edu>
> CC: ncl-talk@ucar.edu
  <ncl-talk@ucar.edu>
>
> Hi Dennis,
> Thanks for the input. The array size is not any larger,
  it just loops
> through finding it many more times.
> I have 6 GB of memory on my computer. I am not sure
  what my variables
> require. But it loops through finding the values it
  requires many times
> before it has a problem. So, there is something
  building up over time,
> but it does not make sense to me because all variables
  are overwritten
> on the next loop.
>
> One thing that I am doing is appending the results for
  each loop to a
> .txt file. Could this be causing some problem?
> I attempted to run the program as you specified below,
  but it does not
> seem to identify the 'top' command.
>
> Thanks very much for your help,
> Lisa
> --------------------------------------------
> On Thu, 3/27/14, Dennis Shea <shea@ucar.edu>
  wrote:
>
> Subject: Re: Fatal Error
  Message
> To: "Lisa Alexander" <lisa_s_a@yahoo.ca>
> Cc: "ncl-talk@ucar.edu"
  <ncl-talk@ucar.edu>
> Received: Thursday, March 27, 2014,
  9:20 AM
>
>
> [1] fatal:NclMalloc Failed:[errno=12]
>
> "I have been running a program on
  my
> computer, but this time
> the case time length is
  very long ..."
>
> I assume this means the
  array size(s) are much
> larger.
>
> You have run out of
  memory.
>
> If the primary variable
  is (say) 6GB and you
> have only 4GB,
> you will get a
  'NclMalloc Failed' error. This
> is regardless
> if you have deleted all
  other variables.
>
> Do you know how much
  memory your computer has
> *available*?
> Do you know how much
  memory your main
> variable(s) need?
>
> [2] One crude approach
>
> %> ncl
  lisaAlexander.ncl >&! z.out
> &
> Then immediately,
> %> top
>
> This will show you the
  memory usage
> of your job.
>
>
> On 3/26/14, 4:12 PM, Lisa Alexander
  wrote:
> > Hello,
> > Please see the problem I have
  been experiencing below
> (noted in a previous post). I have
  tried to delete all
> variables once they are not being used
  and this does not
> work (does not improve the situation
  even slightly).
> > Would it be possible for someone
  to take a look at the
> program to see if there are any
  obvious sinks for computer
> memory?
> > Thanks for your help,
> > Lisa
> >
> > --- On Wed, 3/26/14, Lisa
  Alexander <lisa_s_a@yahoo.ca>
> wrote:
> >
> >> From: Lisa Alexander <lisa_s_a@yahoo.ca>
> >> Subject: Fatal
  Error Message
> >> To: ncl-talk@ucar.edu
> >> Received: Wednesday, March
  26, 2014, 2:56 PM
> >> Hello,
> >> I have been running a program
  on my computer, but
> this time
> >> the case time length is very
  long. When I run it,
> it stops
> >> part way through and gives me
  this message:
> >>
> >> fatal:NclMalloc
  Failed:[errno=12]
> >> Segmentation fault (core
  dumped)
> >>
> >> I must say, whenever I run
  the program (this time
> and other
> >> times), the program runs very
  quickly to begin
> with, but
> >> slows down over time.
> >>
> >> Can somebody let me know what
  the error message
> means?
> >> Thanks,
> >> Lisa
> >>
  _______________________________________________
> >> ncl-talk mailing list
> >> List instructions, subscriber
  options,
> unsubscribe:
> >> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
> >>
> >
  _______________________________________________
> > ncl-talk mailing list
> > List instructions, subscriber
  options, unsubscribe:
> > http://mailman.ucar.edu/mailman/listinfo/ncl-talk
> >
>
>
>
>

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Fri Mar 28 08:25:03 2014