Guess Where? Actor-Supervision for Spatiotemporal Action Localization

Handle URI:
http://hdl.handle.net/10754/627508
Title:
Guess Where? Actor-Supervision for Spatiotemporal Action Localization
Authors:
Escorcia, Victor; Dao, Cuong D.; Jain, Mihir; Ghanem, Bernard ( 0000-0002-5534-587X ) ; Snoek, Cees
Abstract:
This paper addresses the problem of spatiotemporal localization of actions in videos. Compared to leading approaches, which all learn to localize based on carefully annotated boxes on training video frames, we adhere to a weakly-supervised solution that only requires a video class label. We introduce an actor-supervised architecture that exploits the inherent compositionality of actions in terms of actor transformations, to localize actions. We make two contributions. First, we propose actor proposals derived from a detector for human and non-human actors intended for images, which is linked over time by Siamese similarity matching to account for actor deformations. Second, we propose an actor-based attention mechanism that enables the localization of the actions from action class labels and actor proposals and is end-to-end trainable. Experiments on three human and non-human action datasets show actor supervision is state-of-the-art for weakly-supervised action localization and is even competitive to some fully-supervised alternatives.
KAUST Department:
Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division; Electrical Engineering Program; Visual Computing Center (VCC)
Publisher:
arXiv
Issue Date:
5-Apr-2018
ARXIV:
arXiv:1804.01824
Type:
Preprint
Additional Links:
http://arxiv.org/abs/1804.01824v1; http://arxiv.org/pdf/1804.01824v1
Appears in Collections:
Other/General Submission; Electrical Engineering Program; Visual Computing Center (VCC); Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division

Full metadata record

DC FieldValue Language
dc.contributor.authorEscorcia, Victoren
dc.contributor.authorDao, Cuong D.en
dc.contributor.authorJain, Mihiren
dc.contributor.authorGhanem, Bernarden
dc.contributor.authorSnoek, Ceesen
dc.date.accessioned2018-04-16T11:27:42Z-
dc.date.available2018-04-16T11:27:42Z-
dc.date.issued2018-04-05en
dc.identifier.urihttp://hdl.handle.net/10754/627508-
dc.description.abstractThis paper addresses the problem of spatiotemporal localization of actions in videos. Compared to leading approaches, which all learn to localize based on carefully annotated boxes on training video frames, we adhere to a weakly-supervised solution that only requires a video class label. We introduce an actor-supervised architecture that exploits the inherent compositionality of actions in terms of actor transformations, to localize actions. We make two contributions. First, we propose actor proposals derived from a detector for human and non-human actors intended for images, which is linked over time by Siamese similarity matching to account for actor deformations. Second, we propose an actor-based attention mechanism that enables the localization of the actions from action class labels and actor proposals and is end-to-end trainable. Experiments on three human and non-human action datasets show actor supervision is state-of-the-art for weakly-supervised action localization and is even competitive to some fully-supervised alternatives.en
dc.publisherarXiven
dc.relation.urlhttp://arxiv.org/abs/1804.01824v1en
dc.relation.urlhttp://arxiv.org/pdf/1804.01824v1en
dc.rightsArchived with thanks to arXiven
dc.titleGuess Where? Actor-Supervision for Spatiotemporal Action Localizationen
dc.typePreprinten
dc.contributor.departmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Divisionen
dc.contributor.departmentElectrical Engineering Programen
dc.contributor.departmentVisual Computing Center (VCC)en
dc.eprint.versionPre-printen
dc.contributor.institutionQualcomm Technologies, Inc.en
dc.contributor.institutionUniversity of Amsterdamen
dc.identifier.arxividarXiv:1804.01824en
kaust.authorEscorcia, Victoren
kaust.authorDao, Cuong D.en
kaust.authorGhanem, Bernarden
All Items in KAUST are protected by copyright, with all rights reserved, unless otherwise indicated.