Re: possible to set a _FillValue to a string for a numeric type var?

Thank you to Dave and to Mary. Trying to make the _FillValue of a
numeric var be a string is a bit absurd, now that I think about it.

I have turned to a suggestion Dennis gave me (quite a while back) using
ind and have started using this. Here is a sample:

   tData = new( npts, "string", "NULL")
   xTemp = ndtooned(fTemp(0,:,:,:)) ;; create a 3D array (from 4D)
then change that to 1D
   iGood = ind(.not.ismissing(xTemp))
   tData(iGood)=sprintf("%10.12f", xTemp(iGood))
   delete(iGood)

   <snip>
   data = new( npts, "string")

       data = "INSERT INTO iasnowcast " \
         + "(datadatetime, temperature, salinity " \
         + ", elevation, mld, bottomu, bottomv " \
         + ", depth, latitude, longitude, the_geog " \
         + ") VALUES ( (timestamp with time zone '" + newstring + "')" \
         + ", "+tData \
   <snip>

>One resulting INSERT line looks like this:
INSERT INTO iasnowcast (datadatetime, temperature, salinity , elevation,
mld, bottomu, bottomv , depth, latitude, longitude, the_geog ) VALUES (
(timestamp with time zone '2010-06-11T12:00:00-00'), 29.474000930786,
26.291999816895, 0.070000000298, 4.199999809265, NULL, ...

This, possibly, is slowish but is working great on a sampling of the
large data file.

Thanks again,
Donna

On 2/17/11 12:16 PM, Mary Haley wrote:
> Donna,
>
> By defintion, the _FillValue attribute is supposed to be the same type
> as your
> data, so trying to set a string missing value for a float value is not a
> good idea.
> As Dave pointed out, trying to "trick" it may not work as expected.
>
> He's also right about the issue with strings.
>
> A better way to do this might be to write a Fortran subroutine that does the
> formatted writes to a file, and then use WRAPIT to generate a *.so file
> that you can load and call from NCL.
>
> I wrote a small Fortran subroutine that writes formatted data to an ASCII
> file, and then I wrote an NCL script to generate some dummy data and
> call this routine. The Fortran subroutines writes out a blank string if
> the temp value is equal to the missing value.
>
> I'm not a whiz with Fortran, and I imagine the code can be improved!
>
>
> But, hopefully you can use this for your own purposes.
>
> Try this out yourself:
>
> WRAPIT writefile.f
> ncl write.ncl
>
> If this is successful, it will generate a "file.txt" ASCII file that
> looks like this:
>
> HEADER line 1
> HEADER line 2
> time
> 201100
> temp dep lat lon
> 25.9 -100.0 -90.0 -180.0
> -100.0 -90.0 -168.4
> 39.5 -100.0 -90.0 -156.8
> -100.0 -90.0 -145.2
> 28.2 -100.0 -90.0 -133.5
> . . .
> 41.6 1000.0 90.0 180.0
> time
> 201101
> temp dep lat lon
> 19.2 -100.0 -90.0 -180.0
> 28.5 -100.0 -90.0 -168.4
> . . .
>
> Of course, you don't have to write the file this way! I'm just showing
> some possible
> options.
>
> --May
>
>
>
>
>
> On Feb 16, 2011, at 6:46 PM, Dave Allured wrote:
>
>> The use of missing values as function inputs in NCL is often not
>> well defined. The fill value trick may or may not work as you
>> expect. Use the ismissing and "where" functions for well defined
>> results in array situations like this.
>>
>> http://www.ncl.ucar.edu/Document/Functions/Built-in/where.shtml
>>
>> Prototype: where (condtnl_expr, true_value, false_value)
>>
>> Change the last sprintf in your string expression to this (untested):
>>
>> where (ismissing(xTemp), "", sprintf("%10.3f ", xTemp))
>>
>> By the way, you are defining a huge number of individual unique
>> strings in that array expression. NCL may become extremely slow
>> here due to internal string processing.
>>
>> --Dave
>>
>> On 2/16/2011 5:54 PM, donna Cote wrote:
>>> I have this part of my script which has 1D arrays, which are all the
>>> same size, for the Depth (dep3d_1d), Lon, Lat, and Temperature.
>>>
>>> This works to make strings (SQL inserts) of the data, but this includes
>>> any values of Temperature which are "ismissing". That _FillValue for
>>> Temperature is -99 (numeric) and I want it to be NULL (string? ""
>>> right?).
>>>
>>> Can I change the _FillValue like that?
>>> Maybe I should change the _FillValue to some outrageous value like
>>> -9999. It looks like I know how to do that. [ ;) ]
>>> Using -9999 will mean using a follow-up script which removes the -9999
>>> values from the strings. :(
>>>
>>> ncl 2> printVarSummary(xTemp)
>>>
>>>
>>> Variable: xTemp
>>> Type: float
>>> Total Size: 9210168 bytes
>>> 2302542 values
>>> Number of Dimensions: 3
>>> Dimensions and sizes:[Depth | 22] x [Latitude | 261] x [Longitude | 401]
>>> Coordinates:
>>> Depth: [ 0..3500]
>>> Latitude: [18..31]
>>> Longitude: [-98..-78]
>>> Number Of Attributes: 5
>>> Time :252
>>> long_name :Potential Temperature
>>> units :deg C
>>> FORTRAN_format :f8.3
>>> _FillValue :-99
>>> ncl 3> xTemp@_FillValue=-9999
>>> ncl 4> printVarSummary(xTemp)
>>> <snip>
>>> _FillValue :-9999
>>> ncl 5>
>>>
>>> ;;;;;;;;;;;;;;;; snip of script:
>>> data = "INSERT INTO iasnowcast " \
>>> + "(datadatetime, depth, latitude, longitude, the_geog " \
>>> + ", temperature) VALUES ( "+newstring+", " \
>>> + sprintf("%7.16f " , dep3d_1d)+", " \
>>> + sprintf("%7.16f " , lon3d_1d)+", " \
>>> + sprintf("%7.16f " , lat3d_1d)+", ST_MakePoint(" \
>>> + sprintf("%7.16f " , lon3d_1d)+", " \
>>> + sprintf("%7.16f " , lat3d_1d)+", " \
>>> + sprintf("%7.16f " , dep3d_1d)+"), " \
>>> + sprintf("%10.3f ", xTemp)+" );"
>>>
>>>
>>>
>>> _______________________________________________
>>> ncl-talk mailing list
>>> List instructions, subscriber options, unsubscribe:
>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>> _______________________________________________
>> ncl-talk mailing list
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Thu Feb 17 12:59:58 2011