> I need to process large model output files.
>
> Can ncl handle large
> - netCDF files?
> - large variables in a large netCDF file?
> - direct binary files? (fbindirread)
Sylvia pointed out some of the issues with this question, and I'll
bring up some more. Part of the limitation of using NCL to read large
files comes from the limitation on the netCDF software to handle file
offsets larger than 2 Gbytes.
I've included a blurb below from the netCDF FAQ on their web site at:
http://www.unidata.ucar.edu/packages/netcdf/faq.html#lfs
While this mainly addresses creating large netCDF files, I believe the
same concept applies to reading netCDF files:
Q: Is it possible to create netCDF files larger than 2 Gbytes?
A: It is possible to write netCDF files that exceed 2 Gbytes on
platforms that have "Large File Support" (LFS). Such files would
be platform-independent to other LFS platforms, but if you open
such a file on an older platform without LFS, you would expect a
"file too large" error.
There are significant restrictions on the structure of large
netCDF files that result from the 32-bit relative offsets that are
part of the netCDF file format. If you don't use the unlimited
dimension, only one variable can exceed 2 Gbytes in size, but it
can be as large as the underlying file system permits. It must be
the last variable in the dataset, and the offset to the beginning
of this variable must be less than about 2 Gbytes. If you use the
unlimited dimension, any number of record variables may exceed 2
Gbytes in size, as long as the offset of the start of each record
variable within a record is less than about 2 Gbytes. For examples
of both these forms of large netCDF files, see the Large File
Support section in the User's Guide.
To enable LFS for writing large netCDF libraries requires that the
libraries be built with specific combinations of platform-specific
compile flags on some systems.
We have it on our list to try building netCDF with large file support
and seeing how NCL handles it. If some of you have large netCDF files
that you currently cannot read with netCDF, it would be of great help
if we could have access to it for test purposes.
> Have ncl binaries for some 64 bit enabled systems (e.g. Solaris, IRIX,
> HP)
> been compiled with a 64 bit option.
> I mainly use the HP version for this purpose.
>
> Thank you.
> -----
> ENOMOTO, Takeshi, Dr
We have compiled 64-bit versions of NCL under IRIX and AIX, and
these binaries are available at:
http://ngwww.ucar.edu/ncl/download.html
We do not have an HP in-house, so we are unable to provide any
binaries for the HP. Compiling NCL with 64-bit does not automatically
buy you the capability to handle large files, as the above paragraph
explains.
--Mary
_______________________________________________
ncl-talk mailing list
ncl-talk@ucar.edu
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
This archive was generated by hypermail 2b29 : Fri Jul 26 2002 - 10:42:15 MDT