Re: (no subject)

Gang,

In addition to what Wei Huang said, you could divide your input on
any other dimension that is compatible with your processing.

Originally you said you were doing time series analysis. Try
processing the data one level at a time, if your computation will
allow it. This will leave you with complete gridded time series
(t,y,x) in memory, which works well with many math and plot
functions in NCL.

60 Gb / 201 levels = 300 Mb, which is reasonable for a single array
in NCL on most computers.

Also, if you can avoid it, do not read all variables into memory at
the same time. See if you can get just one or two working first.

   fu = addfile ("wrfout_d01_U.nc", "r")
   fv = addfile ("wrfout_d01_V.nc", "r")

   do z = 0, nz-1
     u0=uf->U(:,z,:,:)
     print ("read U for level " + z)
     v0=vf->V(:,z,:,:)
     print ("read V for level " + z)
     [ ... process u and v for one level ... ]
   end do

It is even possible that this method would work with the original
WRF mega file, or with the 1800 time step files that you made, using
addfiles. You might quickly try these two things with the above
loop, before making single variable files.

Dave Allured
CU/CIRES Climate Diagnostics Center (CDC)
http://cires.colorado.edu/science/centers/cdc/
NOAA/ESRL/PSD, Climate Analysis Branch (CAB)
http://www.cdc.noaa.gov/

Wei Huang wrote:
> Gang,
>
> If re-run WRF model, you do not end to change anything from WRF registry.EM.
> You only need to change WRF namelist.input to save WRF output in
> multiple files.
>
> Also, check NCL script to see if you can read in fewer time-levels.
> For example, instead of read in all 1800 time level, try read in one at
> a time, or several at a time.
>
> otherwise, your single variable will have 60G, which very few machine
> has such big memory.
>
> Regards,
>
> Wei Huang
> huangwei_at_ucar.edu <mailto:huangwei_at_ucar.edu>
> Consulting Services Group/CISL
> National Center for Atmospheric Research
> P.O. Box 3000 (1850 Table Mes Dr.)
> Boulder, CO 80307-3000 USA
> (303) 497-8924
>
>
>
> On Dec 8, 2008, at 10:06 PM, Gang Liu wrote:
>
>> Hi Dave and David,
>>
>> Thanks for your advice.
>>
>> I have just tried the option
>> “setfileoption("nc","SuppressClose",False)” David instructed me.
>> However this still does not work. The execution of the NCL script just
>> collapsed without any error information displaying. I doubt it is
>> still related to memory issue, i.e., the NCL script is not able to
>> read variables out of the WRF output data file.
>>
>> The types and dimensions of the data variables such as U, V, W, T,
>> TKE, etc. are float and (1800, 201, 201, 201), which are (nt, nz, ny,
>> nx) respectively.
>>
>> It now looks like that there is only one way remaining to me, which is
>> to break the mega data file up by variable rather than by time step,
>> as suggested by you. Anyway, it has to be broken up.
>>
>> Could you please instruct me how to break it up by variable? I have
>> ever tried it by modifying the Registry.EM file in WRF to reduce the
>> size of the output data file. But the way often easily led to errors
>> of WRF running.
>>
>> Thank you very much.
>>
>> Gang
>>
>>
>>
>> ************************************
>> Gang Liu
>> Department of Atmospheric Sciences
>> Nanjing University
>> Nanjing 210093
>> P. R. China
>> Phone: 86-25-83595185 (o)
>> Email: gangliu_at_nju.edu.cn <mailto:gangliu_at_nju.edu.cn>
>> ************************************
>> _______________________________________________
>> ncl-talk mailing list
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Tue Dec 09 2008 - 12:32:53 MST