Title: Federated Multi-Objective Learning

Abstract: In the first part of this talk, I will give a quick introduction to the NSF AI-EDGE institute led by the Ohio State University. In the second part of this talk, I will dive into a new federated learning paradigm started by my research group called federated multi-objective learning. In recent years, multi-objective optimization (MOO) emerges as a foundational problem underpinning many multi-agent multi-task learning applications. However, existing algorithms in MOO literature remain limited to centralized learning settings, which do not satisfy the distributed nature and data privacy needs of such multi-agent multi-task learning applications. This motivates us to propose a new federated multi-objective learning (FMOL) framework with multiple clients distributively and collaboratively solving an MOO problem while keeping their training data private. Notably, our FMOL framework allows a different set of objective functions across different clients to support a wide range of applications, which advances and generalizes the MOO formulation to the federated learning paradigm for the first time.

For this FMOL framework, we propose two new federated multi-objective optimization (FMOO) algorithms called federated multi-gradient descent averaging (FMGDA) and federated stochastic multi-gradient descent averaging (FSMGDA). Both algorithms allow local updates to significantly reduce communication costs, while achieving the same convergence rates as those of the their algorithmic counterparts in the single-objective federated learning. Our extensive experiments also corroborate the efficacy of our proposed FMOO algorithms.

Bio: Jia (Kevin) Liu is an Assistant Professor in the Dept. of Electrical and Computer Engineering at The Ohio State University (OSU) and an Amazon Visiting Academic (AVA). From Aug. 2017 to Aug. 2020, he was an Assistant Professor in the Dept. of Computer Science at Iowa State University (ISU). He is currently the Managing Director of the NSF AI Institute for Future Edge Networks and Distributed Intelligence (AI-EDGE) at OSU. He is also a faculty investigator of the NSF TRIPODS D4 (Dependable Data-Driven Discovery) Institute at ISU, the NSF ARA Wireless Living Lab PAWR Platform between ISU and OSU, and the Institute of Cybersecurity and Digital Trust (ICDT) at OSU. He received his Ph.D. degree from the Dept. of Electrical and Computer Engineering at Virginia Tech in 2010. His research areas include theoretical machine learning, stochastic network optimization and control, and performance analysis for data analytics infrastructure and cyber-physical systems. Dr. Liu is a senior member of IEEE and a member of ACM. He has received numerous awards at top venues, including IEEE INFOCOM'19 Best Paper Award, IEEE INFOCOM'16 Best Paper Award, IEEE INFOCOM'13 Best Paper Runner-up Award, IEEE INFOCOM'11 Best Paper Runner-up Award, and IEEE ICC'08 Best Paper Award. He has also received multiple honors of long/spotlight presentations at top machine learning conferences, including ICML, NeurIPS, and ICLR. He is an NSF CAREER Award recipient in 2020 and a winner of the Google Faculty Research Award in 2020. He received the LAS Award for Early Achievement in Research at Iowa State University in 2020, and the Bell Labs President Gold Award. Dr. Liu is an Associate Editor for IEEE Transactions on Cognitive Communications and Networking. He has served as a TPC member for numerous top conferences, including ICML, NeurIPS, ICLR, ACM SIGMETRICS, IEEE INFOCOM, and ACM MobiHoc. His research is supported by NSF, AFOSR, AFRL, ONR, Google, and Cisco.