IS IT TIME TO SWISH? COMPARING ACTIVATION FUNCTIONS IN SOLVING THE HELMHOLTZ EQUATION USING PINNS

Abstract

Solving the wave equation numerically constitutes the majority of the computational cost for applications like seismic imaging and full-waveform inversion (FWI). One approach is to solve the frequency-domain Helmholtz equation which allows a reduction in dimensionality as it can be solved per frequency. However, computational challenges with the classical Helmholtz solvers such as the need to invert a large stiffness matrix can make these approaches infeasible for large 3D models or for modeling high frequencies. Moreover, these methods do not have a mechanism to transfer information gained from solving one problem to the next. This becomes a bottleneck for applications like FWI. Therefore, recently an approach based on the emerging paradigm of physics informed neural networks (PINNs) has been proposed to solve the Helmholtz equation. The method has shown promise in addressing several challenges associated with the conventional algorithms. However, the approach still needs further developments to be fully practicable. Foremost amongst the challenges is the slow convergence speed, especially in the presence of sharp heterogeneities in the velocity model. Therefore, we study different activation functions routinely used in the PINN literature, in addition to the swish activation function, which we find to yield superior performance compared to the rest.