Re: memory issue

Mary, thanks for taking your time.

nday = 31 and ntim=24, The lines writing to each ASCII file up to 100000.

On Mon, Sep 30, 2013 at 6:55 PM, Mary Haley <haley@ucar.edu> wrote:

> Hi Cheryl,
>
> Generating lots of strings in an NCL script is known to be a serious
> memory hog. I'm not sure if that's the case, here, however.
>
> How big are your two "do" loops (how big are "nday" and "ntim")? Also, how
> many lines are you generally writing to each ASCII file?
>
> You might be able to use "write_table" instead of "write_matrix", in order
> to get around having to create arrays of formatted string output.
>
> See:
>
> http://www.ncl.ucar.edu/Document/Functions/Built-in/write_table.shtml
>
>
> For example, instead of this code:
>
> lines =
> sprintf("%10.3f",lat)+sprintf("%10.3f",lon)+sprintf("%12.3f",tim)+sprintf("%10.3f",aod_l)+sprintf("%10.3f",aod_o)
> asciiwrite("file.txt",lines)
>
> You could use this code, which writes the values directly to the file
> without creating a bunch of extra strings:
>
> write_table("file.txt","w",[/lat,lon,tim,aod_l,aod_o/],"%10.3f %10.3f
> %12.3f %10.3f %10.3f ")
>
>
> --Mary
>
>
>
> On Sep 27, 2013, at 8:34 AM, cheryl Ma <xiaoyancloud@gmail.com> wrote:
>
> > Hi all,
> >
> > I monitored memory in use on my server and noticed it increased
> gradually when I run a ncl script. The ncl job stopped running when the
> memory is up to the limitation.
> > Here is my script, Any hep on this issue would be very appreciated.
> >
> > Regards,
> > Cheryl
> >
> > =================================
> > begin
> > nday = 31
> > iday_start = 1
> > do nd = 0, nday-1
> > ;
> > dir = "L2/"
> > day = sprinti("%0.2i", nd+iday_start)
> > ;
> > fils = systemfunc("ls
> "+dir+"CER_SSF_Terra-FM1-MODIS_Edition3A_300301.200301"+day+"*.hdf")
> >
> > ntim = dimsizes(fils)
> > do nt = 0,ntim-1
> >
> > lat = f->Colatitude_of_CERES_FOV_at_surface
> > lon = f->Longitude_of_CERES_FOV_at_surface
> > tim = f->Time_of_observation
> > typ = f->Surface_type_index
> > typ_perc = f->Surface_type_percent_coverage
> > clayer=f->Clear_layer_overlap_percent_coverages
> > aod_l = f->PSF_wtd_MOD04_corrected_optical_depth_land_0_550
> > aod_o = f->PSF_wtd_MOD04_effective_optical_depth_average_ocean_0_550
> >
> > nfoot = dimsizes(lat(:))
> >
> > lines = sprinti("%10.0i",ntim)+sprinti("%10.0i",nfoot)
> > asciiwrite("ASCII/200301/zNfoot_2003_01"+day+sprinti("%0.2i",
> nt+1)+".txt",lines)
> >
> > delete(lines)
> > lines =
> sprintf("%10.3f",lat)+sprintf("%10.3f",lon)+sprintf("%12.3f",tim)+sprintf("%10.3f",aod_l)+sprintf("%10.3f",aod_o)
> > asciiwrite("ASCII/200301/zAOD_2003_01"+day+sprinti("%0.2i",
> nt+1)+".txt",lines)
> >
> > opt = True
> > opt@fout = "ASCII/200301/zTYP_2003_01"+day+sprinti("%0.2i",
> nt+1)+".txt"
> > write_matrix (typ, "10i10", opt)
> > ;
> > opt = True
> > opt@fout = "ASCII/200301/zTYC_2003_01"+day+sprinti("%0.2i",
> nt+1)+".txt"
> > write_matrix (typ_perc, "10i10", opt)
> >
> > opt = True
> > opt@fout = "ASCII/200301/zCLA_2003_01"+day+sprinti("%0.2i",
> nt+1)+".txt"
> > write_matrix (clayer, "10f10.3", opt)
> >
> > delete(lines)
> > delete(lat)
> > delete(lon)
> > delete(tim)
> > delete(typ)
> > delete(typ_perc)
> > delete(clayer)
> >
> > end do
> > ;
> > delete(fils)
> > delete(f)
> > end do
> > end
> >
> > _______________________________________________
> > ncl-talk mailing list
> > List instructions, subscriber options, unsubscribe:
> > http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Tue Oct 1 07:35:05 2013