Script crashes while computing WRF diagnostics (SLP, MDBZ, TD2)

NCL Users,

I have a script that runs through a wrfout file and pulls out diagnostics as well as some variables I need to use. The variables are removed without a problem, however when I run this script for larger domains, the diagnostics appear to crash the system. I've been running this on a serial compute node as I've crashed the login node a few times (to the aggravation of other users, I'm sure) while trying to debug this.

Attached is my script as it is right now (ncl_out.ncl). I've tried a coupled of different approaches to running the diagnostics for a domain that is 379(w_e) x 514 (s_n) and 673 timesteps. Using the diagnostic SLP and MDBZ as examples...

Version 1:
Original use was to create an array (ar_3d), save SLP to array at every time step, save array to netCDF file, then overwrite the array for the next variable (MDBZ). This way I was only using one large array (379x514x673) to pull out the variables. This script would crash after ~120 timesteps (not sure exactly where as it doesn't save the array until the end and would not print out the times right before dying). This way was probably the least efficient use of memory.
*********************************************************************************
;create diagnostics array
ar_3d = new( (/ ntimes, south_north, west_east /), float)

;Diagnostics
do time = 0,ntimes-1,1 ;SLP
  slp = wrf_user_getvar(a,"slp",time) ; Sea level pressure in hPa
  ar_3d(time,:,:)=slp
  print(time)
end do
ncf->slp = ar_3d

do time = 0,ntimes-1,1 ;MDBZ
  mdbz = wrf_user_getvar(a,"mdbz",time) ; Simulated radar refl.
  ar_3d(time,:,:)=mdbz
  print(time)
end do
ncf->mdbz = ar_3d
*********************************************************************************

Version 2:
I assumed that this array was getting too large and was choking my memory. My next attempt was to save the output directly to the netCDF file and not use an array in the middle. This script would crash after 234 timesteps (the netCDF file wrote 234 frames and then died while processing the 235th).
*********************************************************************************
;compute diagnostics and export to netCDF file
do time = 0,ntimes-1,1 ;SLP
  ncf->slp(time,:,:) = (/wrf_user_getvar(a,"slp",time)/)
  print(time)
end do

;compute diagnostics and export to netCDF file
do time = 0,ntimes-1,1 ;MDBZ
  ncf->mdbz(time,:,:) = (/wrf_user_getvar(a,"mdbz",time)/)
  print(time)
end do
*********************************************************************************

Version 3:
My last attempt was to bunch all of the diagnostics within the same time loop. I'm not sure how efficient this is for I/O, it didn't seem to take much longer and it saved all 3 diagnostic variables to the netCDF file. This one stopped at 241 timesteps.
*********************************************************************************
;compute diagnostics and export to netCDF file
do time = 0,ntimes-1,1
  ncf->slp(time,:,:) = (/wrf_user_getvar(a,"slp",time)/) ;SLP
  ncf->mdbz(time,:,:) = (/wrf_user_getvar(a,"mdbz",time)/) ;MDBZ
  ncf->td2(time,:,:) = (/wrf_user_getvar(a,"td2",time)/) ;TD2
  print(time)
end do
*********************************************************************************

If anyone has any advice on why my script might be crashing, it would be greatly appreciated. I'm chasing the path down memory allocation (should be a max of 4Gb per node), although I may be way off...

    CPU time : 1882.70 sec.
    Max Memory : 3256 MB
    Max Swap : 11190 MB

    Max Processes : 10
    Max Threads : 12

Also attached is the text outfile (ncl.out) from my last analysis, they're all more or less the same (except the length of analysis). As shown in the outfile, I'm using NCL v. 5.2.0.

Of course, any general critiques on how to make this NCL script more efficient would also be greatly appreciated.

Please let me know if I neglected to mention anything important. Thanks!

-Joe

Joseph B. Zambon
jbzambon@ncsu.edu
NC State University
Department of Marine, Earth and Atmospheric Sciences

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Thu Jan 20 14:42:21 2011