Compositional and Low-shot Understanding of 3D Objects

Abstract

Despite the significant progress in 3D vision in recent years, collecting large amounts of high-quality 3D data remains a challenge. Hence, developing solutions to extract 3D object information efficiently is a significant problem. We aim for an effective shape classification algorithm to facilitate accurate recognition and efficient search of sizeable 3D model databases. This thesis has two contributions in this space: a) a novel meta-learning approach for 3D object recognition and b) propose a new compositional 3D recognition task and dataset. For 3D recognition, we proposed a few-shot semi-supervised meta-learning model based on Pointnet++ representation with a prototypical random walk loss. In particular, we developed the random walk semi-supervised loss that enables fast learning from a few labeled examples by enforcing global consistency over the data manifold and magnetizing unlabeled points around their class prototypes. On the compositional recognition front, we create a large-scale, richly annotated stylized dataset called 3D CoMPaT. This large dataset primarily focuses on stylizing 3D shapes at part-level with compatible materials. We introduce Grounded CoMPaT Recognition as the task of collectively recognizing and grounding compositions of materials on parts of 3D Objects.