Data Assimilation in Chaotic Systems Using Deep Reinforcement Learning

Abstract
Data assimilation (DA) plays a pivotal role in diverse applications, ranging from climate predictions and weather forecasts to trajectory planning for autonomous vehicles. A prime example is the widely used ensemble Kalman filter (EnKF), which relies on linear updates to minimize variance among the ensemble of forecast states. Recent advancements have seen the emergence of deep learning approaches in this domain, primarily within a supervised learning framework. However, the adaptability of such models to untrained scenarios remains a challenge. In this study, we introduce a novel DA strategy that utilizes reinforcement learning (RL) to apply state corrections using full or partial observations of the state variables. Our investigation focuses on demonstrating this approach to the chaotic Lorenz ’63 system, where the agent’s objective is to minimize the root-mean-squared error between the observations and corresponding forecast states. Consequently, the agent develops a correction strategy, enhancing model forecasts based on available system state observations. Our strategy employs a stochastic action policy, enabling a Monte Carlo-based DA framework that relies on randomly sampling the policy to generate an ensemble of assimilated realizations. Results demonstrate that the developed RL algorithm performs favorably when compared to the EnKF. Additionally, we illustrate the agent’s capability to assimilate non-Gaussian data, addressing a significant limitation of the EnKF.

Acknowledgements
Research reported in this publication was supported by the Office of Sponsored Research (OSR) at King Abdullah University of Science and Technology (KAUST) CRG Award #CRG2020-4336 and Virtual Red Sea Initiative Award #REP/1/3268-01-01. The work of E.S.T. was supported in part by NPRP grant # S-0207-200290 from the Qatar National Research Fund (a member of Qatar Foundation).

Publisher
Authorea, Inc.

DOI
10.22541/essoar.170365205.56063528/v1

Additional Links
https://essopenarchive.org/users/640370/articles/695810-data-assimilation-in-chaotic-systems-using-deep-reinforcement-learning?commit=1837eb03d1d25f23126f6a0b29ff614beebb8ee4