Estimation of missing air pollutant data using a spatiotemporal convolutional autoencoder
Permanent link to this recordhttp://hdl.handle.net/10754/676675
MetadataShow full item record
AbstractA key challenge in building machine learning models for time series prediction is the incompleteness of the datasets. Missing data can arise for a variety of reasons, including sensor failure and network outages, resulting in datasets that can be missing significant periods of measurements. Models built using these datasets can therefore be biased. Although various methods have been proposed to handle missing data in many application areas, more air quality missing data prediction requires additional investigation. This study proposes an autoencoder model with spatiotemporal considerations to estimate missing values in air quality data. The model consists of one-dimensional convolution layers, making it flexible to cover spatial and temporal behaviours of air contaminants. This model exploits data from nearby stations to enhance predictions at the target station with missing data. This method does not require additional external features, such as weather and climate data. The results show that the proposed method effectively imputes missing data for discontinuous and long-interval interrupted datasets. Compared to univariate imputation techniques (most frequent, median and mean imputations), our model achieves up to 65% RMSE improvement and 20–40% against multivariate imputation techniques (decision tree, extra-trees, k-nearest neighbours and Bayesian ridge regressors). Imputation performance degrades when neighbouring stations are negatively correlated or weakly correlated.
CitationWardana, I. N. K., Gardner, J. W., & Fahmy, S. A. (2022). Estimation of missing air pollutant data using a spatiotemporal convolutional autoencoder. Neural Computing and Applications. https://doi.org/10.1007/s00521-022-07224-2
SponsorsINK Wardana was supported with a studentship from Indonesia Endowment Fund for Education (LPDP).
This work was supported in part by Indonesia Endowment Fund for Education (LPDP), Ministry of Finance, Republic of Indonesia under grant number Ref: S-1027/LPDP.4/2019
PublisherSpringer Science and Business Media LLC
Except where otherwise noted, this item's license is described as Archived with thanks to Neural Computing and Applications under a Creative Commons license, details at: https://creativecommons.org/licenses/by/4.0