Type
Conference PaperKAUST Department
Visual Computing Center (VCC)Computer, Electrical and Mathematical Science and Engineering (CEMSE) Division
Date
2022-10-10Permanent link to this record
http://hdl.handle.net/10754/682340
Metadata
Show full item recordAbstract
In recent years, supervised or unsupervised learning-based MVS methods achieved excellent performance compared with traditional methods. However, these methods only use the probability volume computed by cost volume regularization to predict reference depths and this manner cannot mine enough information from the probability volume. Furthermore, the unsupervised methods usually try to use two-step or additional inputs for training which make the procedure more complicated. In this paper, we propose the DS-MVSNet, an end-to-end unsupervised MVS structure with the source depths synthesis. To mine the information in probability volume, we creatively synthesize the source depths by splattering the probability volume and depth hypotheses to source views. Meanwhile, we propose the adaptive Gaussian sampling and improved adaptive bins sampling approach that improve the depths hypotheses accuracy. On the other hand, we utilize the source depths to render the reference images and propose depth consistency loss and depth smoothness loss. These can provide additional guidance according to photometric and geometric consistency in different views without additional inputs. Finally, we conduct a series of experiments on the DTU dataset and Tanks $&$ Temples dataset that demonstrate the efficiency and robustness of our DS-MVSNet compared with the state-of-the-art methods.Citation
Li, J., Lu, Z., Wang, Y., Wang, Y., & Xiao, J. (2022). DS-MVSNet: Unsupervised Multi-view Stereo via Depth Synthesis. Proceedings of the 30th ACM International Conference on Multimedia. https://doi.org/10.1145/3503161.3548352Sponsors
This work is supported by the Strategic Priority Research Program of the Chinese Academy of Sciences (No. XDA23090304), the National Natural Science Foundation of China (U2003109, U21A20515, 62102393),the Youth Innovation Promotion Association of the Chinese Academy of Sciences (Y201935), the State Key Laboratory of Robotics and Systems (HIT) (SKLRS-2022-KF-11), and the Fundamental Research Funds for the Central Universities.Publisher
ACMConference/Event name
30th ACM International Conference on MultimediaarXiv
2208.06674Additional Links
https://dl.acm.org/doi/10.1145/3503161.3548352ae974a485f413a2113503eed53cd6c53
10.1145/3503161.3548352