DuLa-Net: A Dual-Projection Network for Estimating Room Layouts From a Single RGB Panorama
Type
Conference PaperKAUST Department
Visual Computing Center (VCC)Computer Science Program
Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
KAUST Grant Number
URF/1/3426-01-01Date
2019Preprint Posting Date
2018-11-29Permanent link to this record
http://hdl.handle.net/10754/660305
Metadata
Show full item recordAbstract
We present a deep learning framework, called DuLa-Net, to predict Manhattan-world 3D room layouts from a single RGB panorama. To achieve better prediction accuracy, our method leverages two projections of the panorama at once, namely the equirectangular panorama-view and the perspective ceiling-view, that each contains different clues about the room layouts. Our network architecture consists of two encoder-decoder branches for analyzing each of the two views. In addition, a novel feature fusion structure is proposed to connect the two branches, which are then jointly trained to predict the 2D floor plans and layout heights. To learn more complex room layouts, we introduce the Realtor360 dataset that contains panoramas of Manhattan-world room layouts with different numbers of corners. Experimental results show that our work outperforms recent state-of-the-art in prediction accuracy and performance, especially in the rooms with non-cuboid layouts.Citation
Yang, S.-T., Wang, F.-E., Peng, C.-H., Wonka, P., Sun, M., & Chu, H.-K. (2019). DuLa-Net: A Dual-Projection Network for Estimating Room Layouts From a Single RGB Panorama. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). doi:10.1109/cvpr.2019.00348Sponsors
The project was funded in part by the KAUST Office of Sponsored Research (OSR) under Award No. URF/1/3426-01-01, and the Ministry of Science and Technology of Taiwan (107-2218-E-007-047- and 107-2221-E-007-088-MY3)Conference/Event name
2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)arXiv
1811.11977Additional Links
https://ieeexplore.ieee.org/document/8953219/https://ieeexplore.ieee.org/document/8953219/
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8953219
ae974a485f413a2113503eed53cd6c53
10.1109/CVPR.2019.00348