Re: logical variable fill_value

From: David Brown <dbrown_at_nyahnyahspammersnyahnyah>
Date: Wed Mar 02 2011 - 13:07:03 MST

copy_VarAtts is an NCL source code procedure. It does not not have special handling for the _FillValue attribute. Since the _FillValue type must agree with type of the variable is belongs to, the float variable's _FillValue attribute value is coerced to the logical type the same as any other value would be. The rule for converting any numeric type to logical is that 0 converts to False and all other values convert to True. Note that during normal assignment of a float variable to a logical variable this copying with coercion of the _FillValue is prevented by NCL internally.

For example:

x = new(3,float)
l = new(3,logical)
printVarSummary(x)
l = x
printVarSummary(l)

gives:

Variable: x
Type: float
Total Size: 12 bytes
            3 values
Number of Dimensions: 1
Dimensions and sizes: [3]
Coordinates:
Number Of Attributes: 1
  _FillValue : 9.96921e+36
ncl 3> l = x
ncl 4> printVarSummary(l)

Variable: l
Type: logical
Total Size: 12 bytes
            3 values
Number of Dimensions: 1
Dimensions and sizes: [3]
Coordinates:
Number Of Attributes: 1
  _FillValue : Missing

But copy_VarAtts is a very simple procedure, and because it directly assigns the attribute values itself, NCL does not, nor should it, prevent the assignment. You could easily create your own version of copy_VarAtts that would catch this problem and handle it. We will discuss whether we should do the same.

Here is the complete code of copy_VarAtts:

undef("copy_VarAtts")
procedure copy_VarAtts(var_from,var_to)
local att_names, i
begin
  att_names =getvaratts(var_from);
    if(.not.all(ismissing(att_names)))
      do i = 0,dimsizes(att_names)-1
         if (isatt(var_to,att_names(i))) then
             delete(var_to@$att_names(i)$) ; var_from att may be diff size/type
         end if
         var_to@$att_names(i)$ = var_from@$att_names(i)$
      end do
  end if
end

Perhaps (untested) add the following after do i = 0,dimsizes(att_names)-1:

     if (islogical(var_to) .and. attnames(i) .eq. "_FillValue") then
         ; don't delete or assign to a logical _FillValue attribute
         continue
     end if

 -dave

On Mar 2, 2011, at 5:38 AM, gibies george wrote:

> Here is some part of my NCL script, where I am using a user defined resource variable "predef" (a logical variable with some attributes attached to it) which is to be given as argument to a function to carry some information about the data to be read from a source file.
>
> I have used copy_VarAtts function to get these attributes from a data variable named "fcstdata"
>
> During this operation the fill value of the logical variable is happens to be "True" .
>
> Why is it so ?
> __________________________________________
> print(predef)
> __________________________________________
> Variable: predef
> Type: logical
> Total Size: 4 bytes
> 1 values
> Number of Dimensions: 1
> Dimensions and sizes: [1]
> Coordinates:
> Number Of Attributes: 15
> datavar : precip
> nosclr : 1
> novctr : 0
> ssnend : 9
> ssnbegin : 6
> season : jjas
> loneast : 360
> lonwest : 0
> latnorth : 30
> latsouth : -30
> region : tropics
> anal_end_year : 2001
> anal_start_year : 1980
> anal : skill
> _FillValue : Missing
> (0) True
>
> __________________________________________
> delete(predef)
> predef = new (1,logical)
> print(predef)
> __________________________________________
> Variable: predef
> Type: logical
> Total Size: 4 bytes
> 1 values
> Number of Dimensions: 1
> Dimensions and sizes: [1]
> Coordinates:
> Number Of Attributes: 1
> _FillValue : Missing
> (0) Missing
>
> __________________________________________
> printVarSummary(fcstdata)
> __________________________________________
> Variable: fcstdata
> Type: float
> Total Size: 304128 bytes
> 76032 values
> Number of Dimensions: 4
> Dimensions and sizes: [time | 22] x [season | 1] x [lat | 24] x [lon | 144]
> Coordinates:
> time: [1980..2001]
> season: [jjas..jjas]
> lat: [-28.75..28.75]
> lon: [1.25..358.75]
> Number Of Attributes: 7
> units : mm/day
> region : tropics
> ssnend : 9
> ssnbegin : 6
> long_name : jjas Means: Average Monthly Rate of Precipitation
> season : jjas
> _FillValue : -9.96921e+36
>
> __________________________________________
> copy_VarAtts(fcstdata,predef)
> print(predef)
> __________________________________________
> Variable: predef
> Type: logical
> Total Size: 4 bytes
> 1 values
> Number of Dimensions: 1
> Dimensions and sizes: [1]
> Coordinates:
> Number Of Attributes: 7
> _FillValue : True
> season : jjas
> long_name : jjas Means: Average Monthly Rate of Precipitation
> ssnbegin : 6
> ssnend : 9
> region : tropics
> units : mm/day
> (0) True
>
> __________________________________________
> predef = True
> print(predef)
> __________________________________________
> Variable: predef
> Type: logical
> Total Size: 4 bytes
> 1 values
> Number of Dimensions: 1
> Dimensions and sizes: [1]
> Coordinates:
> Number Of Attributes: 7
> _FillValue : True
> season : jjas
> long_name : jjas Means: Average Monthly Rate of Precipitation
> ssnbegin : 6
> ssnend : 9
> region : tropics
> units : mm/day
> (0) True
>
>
> On 2 March 2011 16:08, gibies george <gibies@tropmet.res.in> wrote:
>
>
> On 2 March 2011 11:01, gibies george <gibies@tropmet.res.in> wrote:
> Thank you Dave.
>
> Thanking you very much for all your supports.
>
> I am also thankful to Mary and Dennis for the suggestions and support, with great patience through out the long discussion (Both online and offline)
> .
>
>
> On 2 March 2011 06:40, David Brown <dbrown@ucar.edu> wrote:
> Hi Gibies,
>
> There is a problem here and thank you for pointing it out. However, it is not what it might seem at first glance. There is one issue, admittedly somewhat serious: the conversion of the default logical missing value to a string is not working correctly. When you call 'print' with your variable 'full' set to the default missing value, "Missing", as follows:
>
> print("full = " + full)
>
> the value of 'full' gets converted to a string -- incorrectly, it turns out. However, the variable 'full' still has the correct value and the .not. operator is working correctly. You can see this if you print the variable and the expression without converting them to a string. If the first part of your code is changed from:
>
> print("A1 : Defineing full = new(1,logical)")
> full = new(1,logical)
>
> print("full = "+full)
> print(".not.full = "+.not.full)
>
> to
>
> print("A1 : Defining full = new(1,logical)")
> full = new(1,logical)
>
> print(full)
> print(.not.full)
>
> Instead of your output below you get:
>
> (0) A1 : Defining full = new(1,logical)
>
>
> Variable: full
> Type: logical
> Total Size: 4 bytes
> 1 values
> Number of Dimensions: 1
> Dimensions and sizes: [1]
> Coordinates:
> Number Of Attributes: 1
> _FillValue : Missing
> (0) Missing
> (0) Missing
>
> which is correct based on the rule that any operation applied to a missing value yields a missing value.
>
> Actually I considered fill-value of the logical variable as a default state of a switch.
>
> I have used the not operator in that sense.
>
> I was not aware that you you were considering it as missing value same as in the case of all the other data-type.
>
>
> Setting the missing value to True, as you do in case B, inevitably leads to results that seem weird, even though all the results in your B scenario are correct by the rule above. Likewise with scenario C where the missing value is set to False. In most cases, although allowed, it is not advisable to change the missing value of logical variables from the default.
>
> Isn't it possible to have such a different philosophy in the case of logical datatype.
>
> I mean an exception for logical variable alone from "the rule that any operation applied to a missing value yields a missing value", only for the case when fill-value is set explicitly to a value other than missing.
>
> Case D is the same as A and has the same error. Thanks much for pointing this out. I'm not sure how this has managed to slip by for such a long time. But it will have a high priority for getting fixed quickly.
> -dave
>
>
> On Mar 1, 2011, at 9:34 AM, gibies george wrote:
>
>> Here I have analysed 12 different cases of logical variable find its behavior to not operator.
>>
>> My script is attached
>>
>> The output is as follows.
>>
>> Please note the potions with red font colour.
>>
>> I am using NCAR Command Language Version 5.1.1 on Fedora 11 operating system
>>
>> (0) A1 : Defineing full = new(1,logical)
>> (0) full = True
>> (0) .not.full = True
>> (0) case A2 : full = True
>> (0) full = True
>> (0) .not.full = False
>> (0) case A3 : full = False
>> (0) full = False
>> (0) .not.full = True
>> (0) B1 : Defineing full = new(1,logical,True)
>> (0) full = True
>> (0) .not.full = True
>> (0) case B2 : full = True
>> (0) full = True
>> (0) .not.full = True
>> (0) case B3 : full = False
>> (0) full = False
>> (0) .not.full = True
>> (0) C1 : Defineing full = new(1,logical,False)
>> (0) full = False
>> (0) .not.full = False
>> (0) case C2 : full = True
>> (0) full = True
>> (0) .not.full = False
>> (0) case C3 : full = False
>> (0) full = False
>> (0) .not.full = False
>> (0) D1 : Defineing full = new(1,logical)
>> (0) full = True
>> (0) .not.full = True
>> (0) case D2 : full = True
>> (0) full = True
>> (0) .not.full = False
>> (0) case D3 : full = False
>> (0) full = False
>> (0) .not.full = True
>>
>>
>> --
>> Gibies George, CSIR-RF,
>> Climate and Global Modelling Division,
>> Indian Institute of Tropical Meteorology,
>> Dr. Homi Bhabha Road,
>> NCL (P. O.), Pashan,
>> Pune 411008, India.
>>
>> http://sites.google.com/site/gibiesge/
>>
>> Please Think about the environment. Save paper; Save Trees; and don't print this e-mail unless it is necessary.
>> <trial1.ncl>_______________________________________________
>> ncl-talk mailing list
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
>
>
> --
> Gibies George, CSIR-RF,
> Climate and Global Modelling Division,
> Indian Institute of Tropical Meteorology,
> Dr. Homi Bhabha Road,
> NCL (P. O.), Pashan,
> Pune 411008, India.
>
> http://sites.google.com/site/gibiesge/
>
> Please Think about the environment. Save paper; Save Trees; and don't print this e-mail unless it is necessary.
>
>
>
> --
> Gibies George, CSIR-RF,
> Climate and Global Modelling Division,
> Indian Institute of Tropical Meteorology,
> Dr. Homi Bhabha Road,
> NCL (P. O.), Pashan,
> Pune 411008, India.
>
> http://sites.google.com/site/gibiesge/
>
> Please Think about the environment. Save paper; Save Trees; and don't print this e-mail unless it is necessary.
>
>
>
> --
> Gibies George, CSIR-RF,
> Climate and Global Modelling Division,
> Indian Institute of Tropical Meteorology,
> Dr. Homi Bhabha Road,
> NCL (P. O.), Pashan,
> Pune 411008, India.
>
> http://sites.google.com/site/gibiesge/
>
> Please Think about the environment. Save paper; Save Trees; and don't print this e-mail unless it is necessary.
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Wed Mar 2 13:07:11 2011

This archive was generated by hypermail 2.1.8 : Thu Mar 03 2011 - 10:00:25 MST