# Re: grid-cell specific percentiles: sorting without missing values to find percentile

From: Marc Pace Marcella <marcpace_at_nyahnyahspammersnyahnyah>
Date: Fri Mar 22 2013 - 15:30:39 MDT

Hi Mary,

Thank you for the email, much appreciated. I am hoping to get back a 2D array still in the lat-lon.

Essentially, I want to remove all the non-dusty days (vals below 0.1) and then have maps of constant frequency by calculating the grid-cell specific percentile. For example, I would have a 2D lat-lon map dust values for my domain that corresponds to the 90th percentile for each grid-cell. What makes this tricky is that each grid cell will have a different number of days in which there is no dust (vals below 0.1), and I dont want to use/count those (or set them to 0.0) as it will skew the percentile dust value to be too low. I only want to have the percentiles, given there is a dust event. Im not sure if Im confusing in my explanation, but please do let me know if it makes sense. Ive been playing around with the pqsort function which seems to sort my 2D array that has the missing values and places those at the beginning slots of the grid/time step but stlll at a bit of a loss.

Thanks again,

Marc

On Mar 22, 2013, at 5:19 PM, Mary Haley wrote:

> Marc,
>
> When you say you want to remove the missing values from the 2D array, what kind of array do you hope to get back?
>
> If all you want is a single list of numbers that are sorted and have no missing values, then you can convert your 2D array to a 1D array, and use the same method you did before.
>
> UNTESTED:
>
> AOD_all_sum_1d = ndtooned(AOD_all_sum) ; Convert to 1D
> inz = ind(AOD_all_sum_1d .gt. 0.1)
> sort_AOD_nz = AOD_all_sum_1d(inz)
> qsort(sort_AOD_nz)
>
> --Mary
>
> On Mar 20, 2013, at 7:43 PM, Marc Pace Marcella wrote:
>
>> Hi all,
>>
>> I am trying to find different percentiles of occurrence (90th, 75th, 25th, etc) of a certain RCM output variable in a time, lat, lon netcdf file. I realize that the options qsort and dim_pqsort_n do not support ignoring missing values.
>>
>> For a 1D array I am able to use the inz=ind(.not.ismissing(AOD)) command and then the sort_AOD_nz=AOD_nz(inz) to remove all values which are of fill_value, but I am getting tripped up with the added time and 2D grid. Below is the command I am using to mask any AOD events that are under 0.1 and replacing with fill_value:
>>
>> ncl> AOD_ALL_nz=where(AOD_all_sum .gt. 0.1, AOD_all_sum, AOD_all_sum@_FillValue) ;AOD_ALL_nz(time, lat, lon)
>>
>> ncl> AOD_sorted_values=dim_pqsort_n(AOD_ALL_nz_sorted,2,0) ; yields the sorted array increasing in time but with missing_values at the beginning of file for each grid-cell---id like to remove these so they dont count to "skewing" the percentiles to be lower than they actually should be
>>
>>
>>> From here I am unable to figure out how to remove those values which are fill_value when sorting to then find the corresponding percentiles. Essentially I would like to follow a similar approach of the 1D or find the grid-cell specific percentiles some other way.
>>
>> I tried searching the list serve but to no avail. Does anyone have an idea of how to figure out these percentiles (i.e. neglecting events below a certain value) using the sort function or another work-around?
>>
>> Thanks for any help!
>>
>> Marc
>>
>>
>> _______________________________________________
>> ncl-talk mailing list
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Fri Mar 22 15:30:56 2013

This archive was generated by hypermail 2.1.8 : Tue Apr 02 2013 - 21:23:48 MDT