2D-Driven 3D Object Detection in RGB-D Images

Conference Paper

Lahoud, Jean
Ghanem, Bernard

KAUST Department
Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
Electrical Engineering Program
Visual Computing Center (VCC)

Online Publication Date

Print Publication Date


In this paper, we present a technique that places 3D bounding boxes around objects in an RGB-D scene. Our approach makes best use of the 2D information to quickly reduce the search space in 3D, benefiting from state-of-the-art 2D object detection techniques. We then use the 3D information to orient, place, and score bounding boxes around objects. We independently estimate the orientation for every object, using previous techniques that utilize normal information. Object locations and sizes in 3D are learned using a multilayer perceptron (MLP). In the final step, we refine our detections based on object class relations within a scene. When compared to state-of-the-art detection methods that operate almost entirely in the sparse 3D domain, extensive experiments on the well-known SUN RGB-D dataset [29] show that our proposed method is much faster (4.1s per image) in detecting 3D objects in RGB-D images and performs better (3 mAP higher) than the state-of-the-art method that is 4.7 times slower and comparably to the method that is two orders of magnitude slower. This work hints at the idea that 2D-driven object detection in 3D should be further explored, especially in cases where the 3D input is sparse.

Lahoud J, Ghanem B (2017) 2D-Driven 3D Object Detection in RGB-D Images. 2017 IEEE International Conference on Computer Vision (ICCV). Available: http://dx.doi.org/10.1109/ICCV.2017.495.

This work was supported by the King Abdullah University of Science and Technology (KAUST) Office of Sponsored Research.

Institute of Electrical and Electronics Engineers (IEEE)

2017 IEEE International Conference on Computer Vision (ICCV)

Conference/Event Name
16th IEEE International Conference on Computer Vision, ICCV 2017


Additional Links

Permanent link to this record