Name:
Li.etal.2021-SingleImageReconstruction.pdf
Size:
18.29Mb
Format:
PDF
Description:
Accepted manuscript
Type
ArticleAuthors
Li, BoshengKałużny, Jacek
Klein, Jonathan
Michels, Dominik L.
Pałubicki, Wojtek
Benes, Bedrich
Pirk, Sören
KAUST Department
Computer, Electrical and Mathematical Science and Engineering (CEMSE) DivisionComputer Science Program
Visual Computing Center (VCC)
Date
2021-12Permanent link to this record
http://hdl.handle.net/10754/673982
Metadata
Show full item recordAbstract
We introduce a novel method for reconstructing the 3D geometry of botanical trees from single photographs. Faithfully reconstructing a tree from single-view sensor data is a challenging and open problem because many possible 3D trees exist that fit the tree's shape observed from a single view. We address this challenge by defining a reconstruction pipeline based on three neural networks. The networks simultaneously mask out trees in input photographs, identify a tree's species, and obtain its 3D radial bounding volume - our novel 3D representation for botanical trees. Radial bounding volumes (RBV) are used to orchestrate a procedural model primed on learned parameters to grow a tree that matches the main branching structure and the overall shape of the captured tree. While the RBV allows us to faithfully reconstruct the main branching structure, we use the procedural model's morphological constraints to generate realistic branching for the tree crown. This constraints the number of solutions of tree models for a given photograph of a tree. We show that our method reconstructs various tree species even when the trees are captured in front of complex backgrounds. Moreover, although our neural networks have been trained on synthetic data with data augmentation, we show that our pipeline performs well for real tree photographs. We evaluate the reconstructed geometries with several metrics, including leaf area index and maximum radial tree distances.Citation
Li, B., Kałużny, J., Klein, J., Michels, D. L., Pałubicki, W., Benes, B., & Pirk, S. (2021). Learning to reconstruct botanical trees from single images. ACM Transactions on Graphics, 40(6), 1–15. doi:10.1145/3478513.3480525Sponsors
This research was funded in part by National Science Foundation grant #10001387, Functional Proceduralization of 3D Geometric Models. This research was supported by the Foundation for Food and Agriculture Research Grant ID: 602757 to Benes. The content of this publication is solely the responsibility of the authors and does not necessarily represent the official views of the foundation for Food and Agriculture Research. Klein and Michels gratefully acknowledge the baseline funding from of the Computational Sciences Group within KAUST’s Visual Computing CenterJournal
ACM Transactions on GraphicsAdditional Links
https://dl.acm.org/doi/10.1145/3478513.3480525ae974a485f413a2113503eed53cd6c53
10.1145/3478513.3480525