Shape and Reflectance Reconstruction in Uncontrolled Environments by Differentiable Rendering

dc.contributor.authorLi, Rui
dc.contributor.authorZang, Guangmin
dc.contributor.authorQi, Miao
dc.contributor.authorHeidrich, Wolfgang
dc.contributor.departmentComputational Imaging Group
dc.contributor.departmentComputer Science Program
dc.contributor.departmentComputer, Electrical and Mathematical Science and Engineering (CEMSE) Division
dc.contributor.departmentElectrical and Computer Engineering
dc.contributor.departmentElectrical and Computer Engineering Program
dc.contributor.departmentVisual Computing Center (VCC)
dc.date.accessioned2021-10-27T13:00:29Z
dc.date.available2021-10-27T13:00:29Z
dc.date.issued2021-10-25
dc.description.abstractSimultaneous reconstruction of geometry and reflectance properties in uncontrolled environments remains a challenging problem. In this paper, we propose an efficient method to reconstruct the scene's 3D geometry and reflectance from multi-view photography using conventional hand-held cameras. Our method automatically builds a virtual scene in a differentiable rendering system that roughly matches the real world's scene parameters, optimized by minimizing photometric objectives alternatingly and stochastically. With the optimal scene parameters evaluated, photo-realistic novel views for various viewing angles and distances can then be generated by our approach. We present the results of captured scenes with complex geometry and various reflection types. Our method also shows superior performance compared to state-of-the-art alternatives in novel view synthesis visually and quantitatively.
dc.eprint.versionPre-print
dc.identifier.arxivid2110.12975
dc.identifier.urihttp://hdl.handle.net/10754/672983
dc.publisherarXiv
dc.relation.urlhttps://arxiv.org/pdf/2110.12975.pdf
dc.rightsArchived with thanks to arXiv
dc.titleShape and Reflectance Reconstruction in Uncontrolled Environments by Differentiable Rendering
dc.typePreprint
display.details.left<span><h5>Type</h5>Preprint<br><br><h5>Authors</h5><a href="https://repository.kaust.edu.sa/search?query=orcid.id:0000-0001-5332-2167&spc.sf=dc.date.issued&spc.sd=DESC">Li, Rui</a> <a href="https://orcid.org/0000-0001-5332-2167" target="_blank"><img src="https://repository.kaust.edu.sa/server/api/core/bitstreams/82a625b4-ed4b-40c8-865a-d6a5225a26a4/content" width="16" height="16"/></a><br><a href="https://repository.kaust.edu.sa/search?spc.sf=dc.date.issued&spc.sd=DESC&f.author=Zang, Guangmin,equals">Zang, Guangmin</a><br><a href="https://repository.kaust.edu.sa/search?query=orcid.id:0000-0001-5457-4702&spc.sf=dc.date.issued&spc.sd=DESC">Qi, Miao</a> <a href="https://orcid.org/0000-0001-5457-4702" target="_blank"><img src="https://repository.kaust.edu.sa/server/api/core/bitstreams/82a625b4-ed4b-40c8-865a-d6a5225a26a4/content" width="16" height="16"/></a><br><a href="https://repository.kaust.edu.sa/search?query=orcid.id:0000-0002-4227-8508&spc.sf=dc.date.issued&spc.sd=DESC">Heidrich, Wolfgang</a> <a href="https://orcid.org/0000-0002-4227-8508" target="_blank"><img src="https://repository.kaust.edu.sa/server/api/core/bitstreams/82a625b4-ed4b-40c8-865a-d6a5225a26a4/content" width="16" height="16"/></a><br><br><h5>KAUST Department</h5><a href="https://repository.kaust.edu.sa/search?spc.sf=dc.date.issued&spc.sd=DESC&f.department=Computational Imaging Group,equals">Computational Imaging Group</a><br><a href="https://repository.kaust.edu.sa/search?spc.sf=dc.date.issued&spc.sd=DESC&f.department=Computer Science Program,equals">Computer Science Program</a><br><a href="https://repository.kaust.edu.sa/search?spc.sf=dc.date.issued&spc.sd=DESC&f.department=Computer, Electrical and Mathematical Science and Engineering (CEMSE) Division,equals">Computer, Electrical and Mathematical Science and Engineering (CEMSE) Division</a><br><a href="https://repository.kaust.edu.sa/search?spc.sf=dc.date.issued&spc.sd=DESC&f.department=Electrical and Computer Engineering,equals">Electrical and Computer Engineering</a><br><a href="https://repository.kaust.edu.sa/search?spc.sf=dc.date.issued&spc.sd=DESC&f.department=Electrical and Computer Engineering Program,equals">Electrical and Computer Engineering Program</a><br><a href="https://repository.kaust.edu.sa/search?spc.sf=dc.date.issued&spc.sd=DESC&f.department=Visual Computing Center (VCC),equals">Visual Computing Center (VCC)</a><br><br><h5>Date</h5>2021-10-25</span>
display.details.right<span><h5>Abstract</h5>Simultaneous reconstruction of geometry and reflectance properties in uncontrolled environments remains a challenging problem. In this paper, we propose an efficient method to reconstruct the scene's 3D geometry and reflectance from multi-view photography using conventional hand-held cameras. Our method automatically builds a virtual scene in a differentiable rendering system that roughly matches the real world's scene parameters, optimized by minimizing photometric objectives alternatingly and stochastically. With the optimal scene parameters evaluated, photo-realistic novel views for various viewing angles and distances can then be generated by our approach. We present the results of captured scenes with complex geometry and various reflection types. Our method also shows superior performance compared to state-of-the-art alternatives in novel view synthesis visually and quantitatively.<br><br><h5>Publisher</h5><a href="https://repository.kaust.edu.sa/search?spc.sf=dc.date.issued&spc.sd=DESC&f.publisher=arXiv,equals">arXiv</a><br><br><h5>arXiv</h5><a href="https://arxiv.org/abs/2110.12975">2110.12975</a><br><br><h5>Additional Links</h5>https://arxiv.org/pdf/2110.12975.pdf</span>
kaust.personLi, Rui
kaust.personZang, Guangmin
kaust.personQi, Miao
kaust.personHeidrich, Wolfgang
orcid.authorLi, Rui::0000-0001-5332-2167
orcid.authorZang, Guangmin
orcid.authorQi, Miao::0000-0001-5457-4702
orcid.authorHeidrich, Wolfgang::0000-0002-4227-8508
orcid.id0000-0002-4227-8508
orcid.id0000-0001-5457-4702
orcid.id0000-0001-5332-2167
refterms.dateFOA2021-10-27T13:02:52Z
Files
Original bundle
Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
Preprintfile1.pdf
Size:
28.64 MB
Format:
Adobe Portable Document Format
Description:
Pre-print