Manhattan Room Layout Reconstruction from a Single 360 ∘ Image: A Comparative Study of State-of-the-Art Methods

Embargo End Date
2022-02-09

Type
Article

Authors
Zou, Chuhang
Su, Jheng Wei
Peng, Chi Han
Colburn, Alex
Shan, Qi
Wonka, Peter
Chu, Hung Kuo
Hoiem, Derek

KAUST Department
Computer Science Program
Computer, Electrical and Mathematical Science and Engineering (CEMSE) Division
Visual Computing Center (VCC)

Online Publication Date
2021-02-09

Print Publication Date
2021-05

Date
2021-02-09

Submitted Date
2019-10-09

Abstract
Recent approaches for predicting layouts from 360∘ panoramas produce excellent results. These approaches build on a common framework consisting of three steps: a pre-processing step based on edge-based alignment, prediction of layout elements, and a post-processing step by fitting a 3D layout to the layout elements. Until now, it has been difficult to compare the methods due to multiple different design decisions, such as the encoding network (e.g., SegNet or ResNet), type of elements predicted (e.g., corners, wall/floor boundaries, or semantic segmentation), or method of fitting the 3D layout. To address this challenge, we summarize and describe the common framework, the variants, and the impact of the design decisions. For a complete evaluation, we also propose extended annotations for the Matterport3D dataset (Chang et al.: Matterport3d: learning from rgb-d data in indoor environments. arXiv:1709.06158, 2017), and introduce two depth-based evaluation metrics.

Citation
Zou, C., Su, J.-W., Peng, C.-H., Colburn, A., Shan, Q., Wonka, P., … Hoiem, D. (2021). Manhattan Room Layout Reconstruction from a Single $$360^{\circ }$$ Image: A Comparative Study of State-of-the-Art Methods. International Journal of Computer Vision. doi:10.1007/s11263-020-01426-8

Acknowledgements
This research is supported in part by ONR MURI Grant N00014-16-1-2007, iStaging Corp. fund and the Ministry of Science and Technology of Taiwan (108-2218-E-007-050- and 107-2221-E-007-088-MY3). We thank Shang-Ta Yang for providing the source code of DuLa-Net. We thank Cheng Sun for providing the source code of HorizonNet and help run experiments on our provided dataset.

Publisher
Springer Nature

Journal
International Journal of Computer Vision

DOI
10.1007/s11263-020-01426-8

Additional Links
http://link.springer.com/10.1007/s11263-020-01426-8

Permanent link to this record