StructureNet: hierarchical graph networks for 3D shape generation
Type
ArticleKAUST Department
Computer Science ProgramVisual Computing Center (VCC)
Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
KAUST Grant Number
CRG2017-3426Date
2019-11-08Preprint Posting Date
2019-08-01Permanent link to this record
http://hdl.handle.net/10754/660309
Metadata
Show full item recordAbstract
The ability to generate novel, diverse, and realistic 3D shapes along with associated part semantics and structure is central to many applications requiring high-quality 3D assets or large volumes of realistic training data. A key challenge towards this goal is how to accommodate diverse shape variations, including both continuous deformations of parts as well as structural or discrete alterations which add to, remove from, or modify the shape constituents and compositional structure. Such object structure can typically be organized into a hierarchy of constituent object parts and relationships, represented as a hierarchy of n-ary graphs. We introduce StructureNet, a hierarchical graph network which (i) can directly encode shapes represented as such n-ary graphs, (ii) can be robustly trained on large and complex shape families, and (iii) be used to generate a great diversity of realistic structured shape geometries. Technically, we accomplish this by drawing inspiration from recent advances in graph neural networks to propose an order-invariant encoding of n-ary graphs, considering jointly both part geometry and inter-part relations during network training. We extensively evaluate the quality of the learned latent spaces for various shape families and show significant advantages over baseline and competing methods. The learned latent spaces enable several structure-aware geometry processing applications, including shape generation and interpolation, shape editing, or shape structure discovery directly from un-annotated images, point clouds, or partial scans.Citation
Mo, K., Guerrero, P., Yi, L., Su, H., Wonka, P., Mitra, N. J., & Guibas, L. J. (2019). StructureNet. ACM Transactions on Graphics, 38(6), 1–19. doi:10.1145/3355089.3356527Sponsors
This project was supported by a Vannevar Bush Faculty Fellowship, NSF grant RI-1764078, NSF grant CCF-1514305, a Google Research award, an ERC Starting Grant (SmartGeometry StG-2013-335373), ERC PoC Grant (SemanticCity), Google Faculty Awards, Google PhD Fellowships, Royal Society Advanced Newton Fellowship, KAUST OSR number CRG2017-3426 and gifts from Adobe, Autodesk and Qualcomm. We especially thank Kun Liu, Peilang Zhu, Yan Zhang, and Kai Xu for the help preparing binary symmetry hierarchies [Li et al. 2017; Wang et al. 2011a] on PartNet [Mo et al. 2019]. We also thank the anonymous reviewers for their fruitful suggestions.Journal
ACM Transactions on GraphicsAdditional Links
http://dl.acm.org/citation.cfm?doid=3355089.3356527ae974a485f413a2113503eed53cd6c53
10.1145/3355089.3356527