A survey on computational spectral reconstruction methods from RGB to hyperspectral imaging
Type
ArticleKAUST Department
Visual Computing Center (VCC)Computer, Electrical and Mathematical Science and Engineering (CEMSE) Division
Date
2022-07-13Permanent link to this record
http://hdl.handle.net/10754/670139
Metadata
Show full item recordAbstract
Hyperspectral imaging enables many versatile applications for its competence in capturing abundant spatial and spectral information, which is crucial for identifying substances. However, the devices for acquiring hyperspectral images are typically expensive and very complicated, hindering the promotion of their application in consumer electronics, such as daily food inspection and point-of-care medical screening, etc. Recently, many computational spectral imaging methods have been proposed by directly reconstructing the hyperspectral information from widely available RGB images. These reconstruction methods can exclude the usage of burdensome spectral camera hardware while keeping a high spectral resolution and imaging performance. We present a thorough investigation of more than 25 state-of-the-art spectral reconstruction methods which are categorized as prior-based and data-driven methods. Simulations on open-source datasets show that prior-based methods are more suitable for rare data situations, while data-driven methods can unleash the full potential of deep learning in big data cases. We have identified current challenges faced by those methods (e.g., loss function, spectral accuracy, data generalization) and summarized a few trends for future work. With the rapid expansion in datasets and the advent of more advanced neural networks, learnable methods with fine feature representation abilities are very promising. This comprehensive review can serve as a fruitful reference source for peer researchers, thus paving the way for the development of computational hyperspectral imaging.Citation
Zhang, J., Su, R., Fu, Q., Ren, W., Heide, F., & Nie, Y. (2022). A survey on computational spectral reconstruction methods from RGB to hyperspectral imaging. Scientific Reports, 12(1). https://doi.org/10.1038/s41598-022-16223-1Sponsors
This work was supported by the Chinese Academy of Sciences (No. CAS-WX2021-PY-0110, NO. YJKYYQ20180039 and NO. Y70X25A1HY ), and the National Natural Science Foundation of China (NO. 61775219, NO. 61771369 and NO. 61640422). Yunfeng Nie acknowledges the Flemish Fund for Scientific Research (FWO) for supporting her research (No. FWOTM1039). The authors would like to thank those who provide open-sources code for the whole community, such as Sparse Coding, SRUNet, SRMSCNN, HSCNN+ and etc.Publisher
Springer Science and Business Media LLCJournal
Scientific reportsPubMed ID
35831474arXiv
2106.15944Additional Links
https://www.nature.com/articles/s41598-022-16223-1Relations
Is Supplemented By:- [Software]
Title: Intelligent-Imaging-Center/Spectral-Reconstruction. Publication Date: 2021-07-30. github: Intelligent-Imaging-Center/Spectral-Reconstruction Handle: 10754/679846
ae974a485f413a2113503eed53cd6c53
10.1038/s41598-022-16223-1
Scopus Count
Except where otherwise noted, this item's license is described as Archived with thanks to Scientific reports under a Creative Commons license, details at: https://creativecommons.org/licenses/by/4.0
Related articles
- Spectral Representation vis Data-Guided Sparsity for Hyperspectral Image Super-Resolution<sup>†</sup>.
- Authors: Han XH, Sun Y, Wang J, Shi B, Zheng Y, Chen YW
- Issue date: 2019 Dec 7
- Single-shot hyperspectral imaging based on dual attention neural network with multi-modal learning.
- Authors: He T, Zhang Q, Zhou M, Kou T, Shen J
- Issue date: 2022 Mar 14
- Unsupervised super-resolution reconstruction of hyperspectral histology images for whole-slide imaging.
- Authors: Ma L, Rathgeb A, Mubarak H, Tran M, Fei B
- Issue date: 2022 May
- Multimodal Deep Learning and Visible-Light and Hyperspectral Imaging for Fruit Maturity Estimation.
- Authors: Garillos-Manliguez CA, Chiang JY
- Issue date: 2021 Feb 11
- Hyperspectral imaging from a raw mosaic image with end-to-end learning.
- Authors: Fu H, Bian L, Cao X, Zhang J
- Issue date: 2020 Jan 6