Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification
License
http://creativecommons.org/licenses/by/4.0/Type
ArticleAuthors
Chang, JulieSitzmann, Vincent
Dun, Xiong
Heidrich, Wolfgang
Wetzstein, Gordon
KAUST Department
Computer Science ProgramComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
Visual Computing Center (VCC)
Online Publication Date
2018-08-17Print Publication Date
2018-12Date
2018-08-17Abstract
Convolutional neural networks (CNNs) excel in a wide variety of computer vision applications, but their high performance also comes at a high computational cost. Despite efforts to increase efficiency both algorithmically and with specialized hardware, it remains difficult to deploy CNNs in embedded systems due to tight power budgets. Here we explore a complementary strategy that incorporates a layer of optical computing prior to electronic computing, improving performance on image classification tasks while adding minimal electronic computational cost or processing time. We propose a design for an optical convolutional layer based on an optimized diffractive optical element and test our design in two simulations: a learned optical correlator and an optoelectronic two-layer CNN. We demonstrate in simulation and with an optical prototype that the classification accuracies of our optical systems rival those of the analogous electronic implementations, while providing substantial savings on computational cost.Citation
Chang J, Sitzmann V, Dun X, Heidrich W, Wetzstein G (2018) Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification. Scientific Reports 8. Available: http://dx.doi.org/10.1038/s41598-018-30619-y.Acknowledgements
The authors would like to thank Evan Peng for recommending and coordinating fabrication options for the phase mask. The authors also recognize Kevin Ting for rapid prototyping of the rotation mount and Matthew O’Toole, Donald Dansereau, and Joseph Goodman for valuable discussions. J.C. was supported by a National Science Foundation Graduate Research Fellowship. V.S. was supported by a Stanford Graduate Fellowship in Science and Engineering. G.W. was supported by a Sloan Research Fellowship and an NSF CAREER award (IIS 1553333). This project was generously supported by the KAUST Office of Sponsored Research through the Visual Computing Center CCF grant.Publisher
Springer NatureJournal
Scientific ReportsDOI
10.1038/s41598-018-30619-yAdditional Links
https://www.nature.com/articles/s41598-018-30619-yRelations
Is Supplemented By:- [Software]
Title: computational-imaging/opticalCNN: hybrid optical electronic convolutional neural networks. Publication Date: 2018-08-07. github: computational-imaging/opticalCNN Handle: 10754/666954