Re: Missing value behaviour for "esccr" and "escorc"

Thank you for a simple test code.
-------------------------------------
I am not sure how to classify this situation.
It may or may not be a bug.

The normalization standard deviations are calculated
separately for each series [x, y] using the entire series.
This is done once upon initial entry into the subroutine.

Basically, the approach recommended by Chatfield is used.
[Chatfield, The Analysis of Time Series, 1982, Chapman and Hall]
He states that the overall standard deviation of x and y be used.
He offers some statistical arguments to justify doing this.
See the Description section of:
http://www.ncl.ucar.edu/Document/Functions/Built-in/esccr.shtml

An additional unexpected side effect may be correlations
exceeding (+/-) 1 although this is rare.

You can eliminate the behavior via:

print("")
print("*******************************")
print("")
print("===============================")
print("correlation with missing values eliminated")
print("===============================")

ii = ind(.not.ismissing(array1) .and. .not.ismissing(array2) )
nii = dimsizes(ii)
print("nii="+nii)

print("esccr (lag 0): " + esccr(array1(ii), array2(ii), 0))
print("escorc: " + escorc(array2(ii), array2(ii)))

=====

On 2/18/14, 2:41 AM, Evaldsson Martin wrote:
> Hi,
> I'm wondering about the behavior for missing values when computing correlations using "esccr" and "escorc". Using zero lag for "esccr" I expected it should give the same result as "escorc", however, playing around with the attached script it seems to be true only when there are no missing values. The documentation mentions missing values are ok for both "esccr"/"escorc" but from the algorithm on their respective documentation pages it is not clear to me how this difference comes about.
>
> Is there a difference and if so, how can I understand it (to choose the proper routine)?
>
> The attached script reproduces this behavior with both,
>
> ncl_ncarg-6.1.0.Linux_RedHat_x86_64_nodap_gcc444.tar.gz
> ncl_ncarg-6.1.2.Linux_RHEL6.2_x86_64_nodap_gcc446.tar.gz
>
> Best regards,
> Martin
>
>
> --
>
> Martin Evaldsson, PhD
> SMHI / Swedish Meteorological and Hydrological Institute,
> SE-601 76 Norrköping,
> www.smhi.se
>
> Besöksadress/Street address: Folkborgsvägen 17
>
> Tel: +46-11-495 8068
> Fax: +46-11-495 8001
>
>
>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Tue Feb 18 08:34:17 2014