Filtering erroneous data from vertical profiles

From: brianjs _at_nyahnyahspammersnyahnyah <_at_nyahnyahspammersnyahnyah>
Date: Mon May 13 2013 - 09:24:20 MDT

Good Afternoon,

While I have removed the missing data (listed as 999999 in the datasets)
and replaced it with -9999 to have it removed from plotting, there is still
erroneous data (as can be seen in the first 2 plots listed). Most of these
of course, are illustrated in dark red. Comparing these plots to the ones
posted on the ARM data sites for quick viewing (attachments 3, and 4, the
bottom image with the low power settings), these faulty data are left in.
To avoid confusion, please note that my plots reduce the windbarb density
somewhat and incorporate 36 hour observations, concatenated, with x-axis
labeling every 6 hours, and data plotted every 3 hours.

I however, want to remove these random high wind barbs and I have attempted
using the 'where' command to filter some of these points. The idea with
this command is to filter out the missing data and exclude the bad points
at the same time. As such, I had set a threshold for exclusion of data for
the wnd_new variable, where I am essentially excluding the magnitude of the
wind at a certain point. I did it this way instead of setting thresholds
for the u and v components because lower in magnitude u and v components
(both negative and positive) must be excluded in order to nail the bad data
points. I have attached the code (attachment 5) to show specifically what I
am doing. The problem with the u, v component threshold approach is that it
wipes out much of my good data as well. As can be seen by image #5
(attachment 6), the wnd magnitude exclusion approach does not thoroughly
remove the bad data. It just removes the color coding essentially.

Would anyone happen to know a more rigorous and effective way of excluding
these seemingly faulty data points? I know they can be hard coded out, but
that seems to be the least practical method for handling this issue. Any
help would be greatly appreciated.

Lastly, feel free to also send me a reply directly to my email in addition
to the ncl-talk as I do not seem to receive replies in my box (usually
someone in the office also on NCL talk fills me in on the email replies).
Thank you again!

My email (in case it does not show) is

Brian Squitieri
Graduate Assistant
Iowa State University

ncl-talk mailing list
List instructions, subscriber options, unsubscribe:

lamont__wind_profiler_June_15_2012.jpg lamont__wind_profiler_May_24_2007.jpg sgp915rwpwindconC1.a1.20120615.000009.png sgp915rwpwindconC1.a1.20070524.001005.png test_June_15_2012.jpg
Received on Mon May 13 09:24:36 2013

This archive was generated by hypermail 2.1.8 : Wed May 15 2013 - 10:19:28 MDT