3D Room Layout Estimation from One Single RGB Panorama

Abstract

3D Indoor room layout estimation refers to the reconstruction of a 3D layout of a room from a single RGB panoramic image. In this work, we focus on rooms that fit the Manhattan assumption; for each corner of the room, all three composed planes are orthogonal to each other. This paper proposes a new network for indoor room 3D layout estimation based on detection tasks and bin regression, utilizing the encoder-decoder architecture to embed feature and boundary query tokens separately. We also add the deformable attention mechanism to enable our network to extract information from muti-scale features, significantly increasing performance. Besides, several different loss terms have also been experimented with and compared with each other. The proposed network has been trained and tested on the Zillow dataset. Compared with previous SOTA work, our network has surpassed the previous 3D reconstruction accuracy with fewer parameters and fewer training epochs.