Name:
computer vision_1-s2.0-S1077314221000849-main (1).pdf
Size:
1.097Mb
Format:
PDF
Description:
Accepted manuscript
Embargo End Date:
2023-06-24
Type
ArticleAuthors
León Alcázar, JuanBravo, María A.
Jeanneret, Guillaume

Thabet, Ali Kassem

Brox, Thomas
Arbeláez, Pablo
Ghanem, Bernard

KAUST Department
Computer, Electrical and Mathematical Science and Engineering (CEMSE) DivisionElectrical and Computer Engineering Program
VCC Analytics Research Group
Visual Computing Center (VCC)
Date
2021-06-24Preprint Posting Date
2019-04-11Online Publication Date
2021-06-24Print Publication Date
2021-09Embargo End Date
2023-06-24Submitted Date
2020-02-13Permanent link to this record
http://hdl.handle.net/10754/660665
Metadata
Show full item recordAbstract
Instance-level video segmentation requires a solid integration of spatial and temporal information. However, current methods rely mostly on domain-specific information (online learning) to produce accurate instance-level segmentations. We propose a novel approach that relies exclusively on the integration of generic spatio-temporal attention cues. Our strategy, named Multi-Attention Instance Network (MAIN), overcomes challenging segmentation scenarios over arbitrary videos without modelling sequence- or instance-specific knowledge. We design MAIN to segment multiple instances in a single forward pass, and optimize it with a novel loss function that favors class agnostic predictions and assigns instance-specific penalties. We achieve state-of-the-art performance on the challenging Youtube-VOS dataset and benchmark, improving the unseen Jaccard and F-Metric by 6.8% and 12.7% respectively, while operating at real-time (30.3 FPS).Citation
León Alcázar, J., Bravo, M. A., Jeanneret, G., Thabet, A. K., Brox, T., Arbeláez, P., & Ghanem, B. (2021). MAIN: Multi-Attention Instance Network for video segmentation. Computer Vision and Image Understanding, 103240. doi:10.1016/j.cviu.2021.103240Sponsors
This work was partially supported by the King Abdullah University of Science and Technology (KAUST) Office of Sponsored Research, and by the German-Colombian Academic Cooperation between the German Research Foundation (DFG grant BR 3815/9-1) and Universidad de los Andes , Colombia.Publisher
Elsevier BVarXiv
1904.05847Additional Links
https://linkinghub.elsevier.com/retrieve/pii/S1077314221000849ae974a485f413a2113503eed53cd6c53
10.1016/j.cviu.2021.103240