Robust Deep Reinforcement Learning Through Adversarial Attacks and Training : A Survey

Lucas Schott (1, 2) , Josephine Delas (3, 1) , Hatem Hajri (1, 4) , Elies Gherbi (1) , Reda Yaich (1) , Nora Boulahia-Cuppens (3) , Frederic Cuppens (3) , Sylvain Lamprier (2, 5)

1 IRT SystemX
2 MLIA - Machine Learning and Information Access
3 EPM - École Polytechnique de Montréal
4 Safran Tech
5 Marine and Atmospheric Chemistry Division [Miami]

Deep Reinforcement Learning (DRL) is an approach for training autonomous agents across various complex environments. Despite its significant performance in well known environments, it remains susceptible to minor conditions variations, raising concerns about its reliability in real-world applications. To improve usability, DRL must demonstrate trustworthiness and robustness. A way to improve robustness of DRL to unknown changes in the conditions is through Adversarial Training, by training the agent against well suited adversarial attacks on the dynamics of the environment. Addressing this critical issue, our work presents an in-depth analysis of contemporary adversarial attack methodologies, systematically categorizing them and comparing their objectives and operational mechanisms. This classification offers a detailed insight into how adversarial attacks effectively act for evaluating the resilience of DRL agents, thereby paving the way for enhancing their robustness.

Mots clés: Deep Reinforcement Learning Robustness Adversarial Attacks Adversarial Training; Deep Reinforcement Learning; Robustness; Adversarial Attacks; Adversarial Training

Domaines: Apprentissage [cs.LG]; Intelligence artificielle [cs.AI]

Lucas Schott, Josephine Delas, Hatem Hajri, Elies Gherbi, Reda Yaich, et al.. Robust Deep Reinforcement Learning Through Adversarial Attacks and Training : A Survey. 2024. ⟨hal-04521876⟩