Dave and I communicated offline. There is overhead when
creating a compressed file. I have attached a simple test script.
Run on v6.0.0 beta
Bottom line:
(a) Use compression for large files only
(b) Compression can be slow for larger files.
Using compression levels > 1 is probably
not worth the extra time needed to create the file.
Even compression level 1 is slow.
--------------------------------------------------------
netCDF3
4080 Mar 9 13:45 test_nc3.1000.nc
4000080 Mar 9 13:45 test_nc3.1000000.nc
40000080 Mar 9 13:45 test_nc3.10000000.nc
400000080 Mar 9 13:45 test_nc3.100000000.nc
netCDF4: no compression
11215 Mar 9 13:35 test_nc4.cl0.1000.nc <note size overhead
4007820 Mar 9 13:35 test_nc4.cl0.1000000.nc
41956704 Mar 9 13:35 test_nc4.cl0.10000000.nc
402726596 Mar 9 13:35 test_nc4.cl0.100000000.nc
netCDF4: compression level 1
10805 Mar 9 13:45 test_nc4.cl1.1000.nc
3398074 Mar 9 13:45 test_nc4.cl1.1000000.nc
33923350 Mar 9 13:45 test_nc4.cl1.10000000.nc
339070469 Mar 9 13:46 test_nc4.cl1.100000000.nc
netCDF4: compression level 5
10789 Mar 9 13:33 test_nc4.cl5.1000.nc
3365173 Mar 9 13:33 test_nc4.cl5.1000000.nc
33597009 Mar 9 13:33 test_nc4.cl5.10000000.nc
335844543 Mar 9 13:34 test_nc4.cl5.100000000.nc
Elapsed time to create each file:
=====> Wall Clock Elapsed Time: test_nc3.1000.nc: 0 seconds
=====> Wall Clock Elapsed Time: test_nc3.1000000.nc: 0 seconds
=====> Wall Clock Elapsed Time: test_nc3.10000000.nc: 0 seconds
=====> Wall Clock Elapsed Time: test_nc3.100000000.nc: 3 seconds
=====> Wall Clock Elapsed Time: test_nc4.cl1.1000.nc: 0 seconds
=====> Wall Clock Elapsed Time: test_nc4.cl1.1000000.nc: 1 seconds
=====> Wall Clock Elapsed Time: test_nc4.cl1.10000000.nc: 3 seconds
=====> Wall Clock Elapsed Time: test_nc4.cl1.100000000.nc: 31 seconds **
=====> Wall Clock Elapsed Time: test_nc4.cl5.1000.nc: 0 seconds
=====> Wall Clock Elapsed Time: test_nc4.cl5.1000000.nc: 0 seconds
=====> Wall Clock Elapsed Time: test_nc4.cl5.10000000.nc: 4 seconds
=====> Wall Clock Elapsed Time: test_nc4.cl5.100000000.nc: 43 seconds **
On 03/09/2011 11:14 AM, David Brown wrote:
> I cannot reproduce this error with ncl 6.0.0-beta either on mac 10.6.x
> or on an x86_64 Linux machine. Can you let us know more details about
> your environment? Which binary? Built from source, etc?
> -dave
>
> On Mar 9, 2011, at 10:59 AM, Robert Schuster wrote:
>
>> Hello,
>>
>> is it not possible to create compressed netCDF-Files with NCL 6.0.0-beta?
>>
>> This small script is leading to a segmentation fault:
>>
>> begin
>> setfileoption("nc", "Format", "NetCDF4Classic")
>> setfileoption("nc", "CompressionLevel", 5)
>>
>> var = new (1000,float)
>> outfile = addfile("test.nc","c")
>> outfile->var = var
>> delete(outfile)
>> end
>>
>> Thank you!
>> Robert
>>
>> _______________________________________________
>> ncl-talk mailing list
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
-- ====================================================== Dennis J. Shea tel: 303-497-1361 | P.O. Box 3000 fax: 303-497-1333 | Climate Analysis Section | Climate & Global Dynamics Div. | National Center for Atmospheric Research | Boulder, CO 80307 | USA email: shea 'at' ucar.edu | ======================================================
_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
This archive was generated by hypermail 2.1.8 : Thu Mar 10 2011 - 21:44:37 MST