TRIP: An Interactive Retrieving-Inferring Data Imputation Approach
KAUST DepartmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
Computer Science Program
Online Publication Date2015-03-09
Print Publication Date2015-09-01
Permanent link to this recordhttp://hdl.handle.net/10754/630001
MetadataShow full item record
AbstractData imputation aims at filling in missing attribute values in databases. Most existing imputation methods to string attribute values are inferring-based approaches, which usually fail to reach a high imputation recall by just inferring missing values from the complete part of the data set. Recently, some retrieving-based methods are proposed to retrieve missing values from external resources such as the World Wide Web, which tend to reach a much higher imputation recall, but inevitably bring a large overhead by issuing a large number of search queries. In this paper, we investigate the interaction between the inferring-based methods and the retrieving-based methods. We show that retrieving a small number of selected missing values can greatly improve the imputation recall of the inferring-based methods. With this intuition, we propose an inTeractive Retrieving-Inferring data imPutation approach (TRIP), which performs retrieving and inferring alternately in filling in missing attribute values in a data set. To ensure the high recall at the minimum cost, TRIP faces a challenge of selecting the least number of missing values for retrieving to maximize the number of inferable values. Our proposed solution is able to identify an optimal retrieving-inferring scheduling scheme in deterministic data imputation, and the optimality of the generated scheme is theoretically analyzed with proofs. We also analyze with an example that the optimal scheme is not feasible to be achieved in τ-constrained stochastic data imputation (τ-SDI), but still, our proposed solution identifies an expected-optimal scheme in τ-SDI. Extensive experiments on four data collections show that TRIP retrieves on average 20 percent missing values and achieves the same high recall that was reached by the retrieving-based approach.
CitationLi Z, Qin L, Cheng H, Zhang X, Zhou X (2015) TRIP: An Interactive Retrieving-Inferring Data Imputation Approach. IEEE Transactions on Knowledge and Data Engineering 27: 2550–2563. Available: http://dx.doi.org/10.1109/TKDE.2015.2411276.
SponsorsThis work was supported in part by the Natural Science Foundation of China under Grants 61472263, 61402313, 61303019, the Australian Research Council under Grants DP140103171, DE140100999, the Hong Kong Research Grants Council (General Research Fund (GRF) Project CUHK 411211), the Chinese University of Hong Kong (Direct Grants 4055015, 4055048), and the King Abdullah University of Science and Technology.