Speaker

Dr. Yanbing Mao, Wayne State University

Abstract

The recent incidents due to the deployment of machine learning (ML) models overshadow the revolutionizing potential of ML, especially for safety-critical autonomous systems. Developing safe ML is thus more vital today. In the ML community, deep reinforcement learning (DRL) has demonstrated breakthroughs in sequential decision-making in broad areas, ranging from autonomous driving to games. This motivates us to develop a DRL-based safe learning framework. However, applying DRL to safety-critical systems remains challenging. It has a deep root to the control policy of DRL being parameterized by deep neural networks (DNN), whose behaviors are hard to predict and verify, raising a safety concern. Meanwhile, it was recently discovered that purely data-driven DNN applied to physical systems can infer relations violating physics laws and sometimes lead to catastrophic consequences (e.g., data-driven blackout). This thus induces another safety concern for DRL that adopts purely data-driven DNN. To address the safety concerns, we proposed the Phy-DRL: a physics-regulated deep reinforcement learning framework with particularly enhanced safety assurance. The Phy-DRL exhibits 1) a provable safety guarantee, and 2) strict compliance of critic and actor networks with physics knowledge about the action value function and control policy. The experiments on quadruped robots validate our results. The talk will present three invariant-embedding principles for Phy-DRL design: i) residual action policy (i.e., integrating data-driven-DRL action policy and physics-model-based action policy), ii) safety-embedded reward, and iii) physics-model-guided neural network (NN) editing.

Biography

Dr. Mao is an Assistant Professor with the Engineering Technology Division, at Wayne State University, MI, USA. He received his Ph.D. degree in Electrical and Computer Engineering from the State University of New York at Binghamton in 2019. From 2019 to 2022, he was a Postdoctoral Researcher at the University of Illinois Urbana–Champaign. His current research focuses on Safe Physics-AI, with applications to self-driving vehicles and quadruped robots. Dr. Mao is leading an international research group of Physics-AI, with over 20 members from the US, Germany, and Hong Kong. NSF has funded his research team on Physics-AI for 1.2 mln.