What Do I Annotate Next? An Empirical Study of Active Learning for Action Localization
KAUST DepartmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
Electrical Engineering Program
Visual Computing Center (VCC)
KAUST Grant NumberOSR-CRG2017-3405
Online Publication Date2018-10-06
Print Publication Date2018
Permanent link to this recordhttp://hdl.handle.net/10754/630638
MetadataShow full item record
AbstractDespite tremendous progress achieved in temporal action localization, state-of-the-art methods still struggle to train accurate models when annotated data is scarce. In this paper, we introduce a novel active learning framework for temporal localization that aims to mitigate this data dependency issue. We equip our framework with active selection functions that can reuse knowledge from previously annotated datasets. We study the performance of two state-of-the-art active selection functions as well as two widely used active learning baselines. To validate the effectiveness of each one of these selection functions, we conduct simulated experiments on ActivityNet. We find that using previously acquired knowledge as a bootstrapping source is crucial for active learners aiming to localize actions. When equipped with the right selection function, our proposed framework exhibits significantly better performance than standard active learning strategies, such as uncertainty sampling. Finally, we employ our framework to augment the newly compiled Kinetics action dataset with ground-truth temporal annotations. As a result, we collect Kinetics-Localization, a novel large-scale dataset for temporal action localization, which contains more than 15K YouTube videos.
CitationHeilbron FC, Lee J-Y, Jin H, Ghanem B (2018) What Do I Annotate Next? An Empirical Study of Active Learning for Action Localization. Lecture Notes in Computer Science: 212–229. Available: http://dx.doi.org/10.1007/978-3-030-01252-6_13.
SponsorsThis publication is based upon work supported by the King Abdullah University of Science and Technology (KAUST) Office of Sponsored Research (OSR) under Award No. OSR-CRG2017-3405.
Conference/Event name15th European Conference on Computer Vision, ECCV 2018