Robust bivariate error detection in skewed data with application to historical radiosonde winds

Handle URI:
http://hdl.handle.net/10754/623903
Title:
Robust bivariate error detection in skewed data with application to historical radiosonde winds
Authors:
Sun, Ying ( 0000-0001-6703-4270 ) ; Hering, Amanda S.; Browning, Joshua M.
Abstract:
The global historical radiosonde archives date back to the 1920s and contain the only directly observed measurements of temperature, wind, and moisture in the upper atmosphere, but they contain many random errors. Most of the focus on cleaning these large datasets has been on temperatures, but winds are important inputs to climate models and in studies of wind climatology. The bivariate distribution of the wind vector does not have elliptical contours but is skewed and heavy-tailed, so we develop two methods for outlier detection based on the bivariate skew-t (BST) distribution, using either distance-based or contour-based approaches to flag observations as potential outliers. We develop a framework to robustly estimate the parameters of the BST and then show how the tuning parameter to get these estimates is chosen. In simulation, we compare our methods with one based on a bivariate normal distribution and a nonparametric approach based on the bagplot. We then apply all four methods to the winds observed for over 35,000 radiosonde launches at a single station and demonstrate differences in the number of observations flagged across eight pressure levels and through time. In this pilot study, the method based on the BST contours performs very well.
KAUST Department:
Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
Citation:
Sun Y, Hering AS, Browning JM (2017) Robust bivariate error detection in skewed data with application to historical radiosonde winds. Environmetrics 28: e2431. Available: http://dx.doi.org/10.1002/env.2431.
Publisher:
Wiley-Blackwell
Journal:
Environmetrics
KAUST Grant Number:
OSR-2015-CRG4-2582
Issue Date:
18-Jan-2017
DOI:
10.1002/env.2431
Type:
Article
ISSN:
1180-4009
Sponsors:
The authors would like to thank Adelchi Azzalini, Douglas Nychka, SteveWorley, and Joey Comeaux for discussions and ideas that helped to direct and focus this work. In addition, we are grateful to two referees for providing useful comments on the paper. This publication is based upon work supported by the King Abdullah University of Science and Technology (KAUST) Office of Sponsored Research (OSR) under Award OSR-2015-CRG4-2582.
Additional Links:
http://onlinelibrary.wiley.com/doi/10.1002/env.2431/abstract
Appears in Collections:
Articles; Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division

Full metadata record

DC FieldValue Language
dc.contributor.authorSun, Yingen
dc.contributor.authorHering, Amanda S.en
dc.contributor.authorBrowning, Joshua M.en
dc.date.accessioned2017-05-31T11:23:12Z-
dc.date.available2017-05-31T11:23:12Z-
dc.date.issued2017-01-18en
dc.identifier.citationSun Y, Hering AS, Browning JM (2017) Robust bivariate error detection in skewed data with application to historical radiosonde winds. Environmetrics 28: e2431. Available: http://dx.doi.org/10.1002/env.2431.en
dc.identifier.issn1180-4009en
dc.identifier.doi10.1002/env.2431en
dc.identifier.urihttp://hdl.handle.net/10754/623903-
dc.description.abstractThe global historical radiosonde archives date back to the 1920s and contain the only directly observed measurements of temperature, wind, and moisture in the upper atmosphere, but they contain many random errors. Most of the focus on cleaning these large datasets has been on temperatures, but winds are important inputs to climate models and in studies of wind climatology. The bivariate distribution of the wind vector does not have elliptical contours but is skewed and heavy-tailed, so we develop two methods for outlier detection based on the bivariate skew-t (BST) distribution, using either distance-based or contour-based approaches to flag observations as potential outliers. We develop a framework to robustly estimate the parameters of the BST and then show how the tuning parameter to get these estimates is chosen. In simulation, we compare our methods with one based on a bivariate normal distribution and a nonparametric approach based on the bagplot. We then apply all four methods to the winds observed for over 35,000 radiosonde launches at a single station and demonstrate differences in the number of observations flagged across eight pressure levels and through time. In this pilot study, the method based on the BST contours performs very well.en
dc.description.sponsorshipThe authors would like to thank Adelchi Azzalini, Douglas Nychka, SteveWorley, and Joey Comeaux for discussions and ideas that helped to direct and focus this work. In addition, we are grateful to two referees for providing useful comments on the paper. This publication is based upon work supported by the King Abdullah University of Science and Technology (KAUST) Office of Sponsored Research (OSR) under Award OSR-2015-CRG4-2582.en
dc.publisherWiley-Blackwellen
dc.relation.urlhttp://onlinelibrary.wiley.com/doi/10.1002/env.2431/abstracten
dc.subjectoutliersen
dc.subjectradiosonde windsen
dc.subjectskewed multivariate distributionsen
dc.titleRobust bivariate error detection in skewed data with application to historical radiosonde windsen
dc.typeArticleen
dc.contributor.departmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Divisionen
dc.identifier.journalEnvironmetricsen
dc.contributor.institutionDepartment of Statistical Science; Baylor University; Waco TX USAen
dc.contributor.institutionApplied Mathematics and Statistics; Colorado School of Mines; Golden CO USAen
kaust.authorSun, Yingen
kaust.grant.numberOSR-2015-CRG4-2582en
All Items in KAUST are protected by copyright, with all rights reserved, unless otherwise indicated.