Adressing a variable > 2 Gb < 4 Gb

Hello all,

In March 2008, I have asked that question, about the possibility to have
greater than 2 Gb variable, but less than 4 Gb (see emails below).

I tried again today with NCL 5.1.1
ncl_ncarg-5.1.1.Linux_x86_64_gcc412.tar.gz, and it still doesn't work. Any
plan yet for an NCL release which will support it?

I was trying to open a variable stored in 87 files of 28 Mb each.

Here is the valgrind output:

==11968== Source and destination overlap in memcpy(0xE748EE0, 0xE749030,
27976520)
==11968== at 0x4907456: memcpy (mc_replace_strmem.c:402)
==11968== by 0x5E0382: CallLIST_READ_FILEVAR_OP (in
/contrex/local64/ncl/bin/ncl)
==11968== by 0x5E09C4: _NclExecute (in /contrex/local64/ncl/bin/ncl)
==11968== by 0x5E6A8E: _NclFuncCallOp (in /contrex/local64/ncl/bin/ncl)
==11968== by 0x5DABAE: CallFUNC_CALL_OP (in /contrex/local64/ncl/bin/ncl)
==11968== by 0x5E0884: _NclExecute (in /contrex/local64/ncl/bin/ncl)
==11968== by 0x51FC68: yyparse (in /contrex/local64/ncl/bin/ncl)
==11968== by 0x51B4CD: main (in /contrex/local64/ncl/bin/ncl)
==11968==
==11968== Invalid write of size 1
==11968== at 0x49074FC: memcpy (mc_replace_strmem.c:402)
==11968== by 0x5E0382: CallLIST_READ_FILEVAR_OP (in
/contrex/local64/ncl/bin/ncl)
==11968== by 0x5E09C4: _NclExecute (in /contrex/local64/ncl/bin/ncl)
==11968== by 0x5E6A8E: _NclFuncCallOp (in /contrex/local64/ncl/bin/ncl)
==11968== by 0x5DABAE: CallFUNC_CALL_OP (in /contrex/local64/ncl/bin/ncl)
==11968== by 0x5E0884: _NclExecute (in /contrex/local64/ncl/bin/ncl)
==11968== by 0x51FC68: yyparse (in /contrex/local64/ncl/bin/ncl)
==11968== by 0x51B4CD: main (in /contrex/local64/ncl/bin/ncl)
==11968== Address 0xe748ee0 is 0 bytes after a block of size 28,053,168
alloc'd
==11968== at 0x4904E57: malloc (vg_replace_malloc.c:207)
==11968== by 0x568182: NclMalloc (Memory.c:57)
==11968== by 0x5754AB: MyFileReadVarValue (in
/contrex/local64/ncl/bin/ncl)
==11968== by 0x578095: FileReadVar (in /contrex/local64/ncl/bin/ncl)
==11968== by 0x554444: _NclFileReadVar (in /contrex/local64/ncl/bin/ncl)
==11968== by 0x5DFB01: CallLIST_READ_FILEVAR_OP (in
/contrex/local64/ncl/bin/ncl)
==11968== by 0x5E09C4: _NclExecute (in /contrex/local64/ncl/bin/ncl)
==11968== by 0x5E6A8E: _NclFuncCallOp (in /contrex/local64/ncl/bin/ncl)
==11968== by 0x5DABAE: CallFUNC_CALL_OP (in /contrex/local64/ncl/bin/ncl)
==11968== by 0x5E0884: _NclExecute (in /contrex/local64/ncl/bin/ncl)
==11968== by 0x51FC68: yyparse (in /contrex/local64/ncl/bin/ncl)
==11968== by 0x51B4CD: main (in /contrex/local64/ncl/bin/ncl)

Best regards,

Christian Page

=======================

Hello Rick,

I would be quite interested in testing it ! As soon as you have a working
version, please send it to me and I will test it.
The servers I am working on are :
Linux 2.6.22.14-72.fc6 #1 SMP Wed Nov 21 14:10:25 EST 2007 x86_64 x86_64
x86_64 GNU/Linux
for the 64-bit linux

and :
Linux 2.6.20-1.2320.fc5smp #1 SMP Tue Jun 12 19:40:16 EDT 2007 i686 i686
i386 GNU/Linux
for 32-bit linux.

Christian

2008/3/17, Rick Grubin <grubin_at_ucar.edu>:
*> *
*> Hello Christian *
*> *
*> *
*> > I would need to address a variable past the 2 Gb memory limit. The *
*> > variable is read off from addfiles using addfiles_GetVar on several *
*> > separate small NetCDF classic files. Is it possible to achieve *
*> > this ? I have both 32-bit and 64-bit Linux machines I can use. *
*> *
*> *
*> At the moment, NCL does not handle variables > 2Gb, regardless of the *
*> host you are on. This is a limitation internal to NCL, based on *
*> its current implementation. *
*> *
*> That said, we have been working on upgrading NCL to be able to *
*> address a variable > 2Gb. We have made good progress and hope to *
*> present to users a version for testing very soon. Would you be *
*> interested in helping us test? *
*> *
*> ---- *
*> *
*> Rick Grubin NCAR/TDD/CISL/VETS *
*> Visualization + Enabling Technologies *
*> grubin_at_ucar.edu 303.497.1832 *
*> *
*> *

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Wed Sep 30 2009 - 08:54:30 MDT