Speaker: Yue Xing, Department of Statistics and Probability, Michigan State University

Time: Thursday, March 12, from 2:30 PM to 3:30 PM

Location for in-person participants: 1146 FAB 

Zoom link for online audience: https://wayne-edu.zoom.us/j/92845590121?pwd=CpRA5Wa5gzSMn2xiVkR2abD83O5nrH.1

Title: Divergence-Based Reinforcement Learning Algorithms for General LLM Alignment

Abstract: Alignment is the final stage in the training pipeline of large language models (LLMs), aimed at instilling capabilities beyond language comprehension and instruction-following that are learned during pretraining and instruction fine-tuning. While various studies have developed and enhanced alignment algorithms for different tasks and data, there lacks a unified understanding on how and why these algorithms work. In this work, we provide a unified analysis framework from a divergence-based perspective and consider general alignment settings, such as reinforcement learning with verifiable rewards (RLVR) and Preference Alignment (PA). Within this unified framework, we identify some potential issues in existing alignment methods, such as the widely used Group Relative Policy Optimization (GRPO). We further propose f-GRPO, a class of on-policy reinforcement learning, and f-Hybrid Alignment Loss (f-HAL), a hybrid on/off policy objectives, for general LLM alignment based on variational representation of f-divergences. 

Bio: Yue Xing is an Assistant Professor in the Department of Statistics and Probability at Michigan State University. She received her Ph.D. from Purdue University. Her research focuses on trustworthy artificial intelligence, with an emphasis on the theoretical understanding of Transformers and deep learning models. She also conducts interdisciplinary research in AI for education and has led the development and deployment of AI-driven assessment and learning systems across K–12, medical, and higher education settings.

______________________________________________________________________

Speaker: Yue Xing, Department of Statistics and Probability, Michigan State University

Time: Thursday, March 12, from 2:30 PM to 3:30 PM

Location for in-person participants: 1146 FAB 

Zoom link for online audience: https://wayne-edu.zoom.us/j/92845590121?pwd=CpRA5Wa5gzSMn2xiVkR2abD83O5nrH.1

Title: Divergence-Based Reinforcement Learning Algorithms for General LLM Alignment

Abstract: Alignment is the final stage in the training pipeline of large language models (LLMs), aimed at instilling capabilities beyond language comprehension and instruction-following that are learned during pretraining and instruction fine-tuning. While various studies have developed and enhanced alignment algorithms for different tasks and data, there lacks a unified understanding on how and why these algorithms work. In this work, we provide a unified analysis framework from a divergence-based perspective and consider general alignment settings, such as reinforcement learning with verifiable rewards (RLVR) and Preference Alignment (PA). Within this unified framework, we identify some potential issues in existing alignment methods, such as the widely used Group Relative Policy Optimization (GRPO). We further propose f-GRPO, a class of on-policy reinforcement learning, and f-Hybrid Alignment Loss (f-HAL), a hybrid on/off policy objectives, for general LLM alignment based on variational representation of f-divergences. 

Bio: Yue Xing is an Assistant Professor in the Department of Statistics and Probability at Michigan State University. She received her Ph.D. from Purdue University. Her research focuses on trustworthy artificial intelligence, with an emphasis on the theoretical understanding of Transformers and deep learning models. She also conducts interdisciplinary research in AI for education and has led the development and deployment of AI-driven assessment and learning systems across K–12, medical, and higher education settings.

______________________________________________________________________

AIDaS: CAD Seminar Series

Advancing Knowledge, Innovation, and Collaboration in Computation, AI, and Data Science (CAD)

The CAD Seminar Series is a primary seminar series at Wayne State University’s Institute for AI and Data Science (AIDaS). It is a dedicated platform for advancing knowledge, fostering innovation, and promoting collaboration across the fields of Computation, Artificial Intelligence, and Data Science. This series brings together leading experts, researchers, and professionals to explore the latest developments, tackle emerging challenges, and drive forward-thinking solutions at the convergence of these critical disciplines.

Objectives:

• Advance Knowledge: Share cutting-edge research and insights that push the boundaries of what is known in CAD.

• Foster Innovation: Encourage the development of novel ideas and solutions through interdisciplinary dialogue and creative thinking.

• Promote Collaboration: Unite expertise across disciplines and build bridges between academia, industry, and government to address complex problems and create opportunities for joint ventures.

Target Audience: The CAD Seminar Series is designed for a diverse audience, including faculty, researchers, students, and professionals in Computation, AI, Data Science, and related fields. It serves as a forum for exchanging ideas, networking, and contributing to the growth of these rapidly evolving areas. We highly recommend in-person attendance to enhance engagement and networking opportunities with speakers and fellow participants.

Call for Participation: We welcome contributions from researchers, practitioners, and students. Whether presenting your work, participating in discussions, or attending as a learner, your involvement is crucial to the success of this collaborative initiative. 

Advancing Knowledge, Innovation, and Collaboration in Computation, AI, and Data Science (CAD)

The CAD Seminar Series is a primary seminar series at Wayne State University’s Institute for AI and Data Science (AIDaS). It is a dedicated platform for advancing knowledge, fostering innovation, and promoting collaboration across the fields of Computation, Artificial Intelligence, and Data Science. This series brings together leading experts, researchers, and professionals to explore the latest developments, tackle emerging challenges, and drive forward-thinking solutions at the convergence of these critical disciplines.

Objectives:

• Advance Knowledge: Share cutting-edge research and insights that push the boundaries of what is known in CAD.

• Foster Innovation: Encourage the development of novel ideas and solutions through interdisciplinary dialogue and creative thinking.

• Promote Collaboration: Unite expertise across disciplines and build bridges between academia, industry, and government to address complex problems and create opportunities for joint ventures.

Target Audience: The CAD Seminar Series is designed for a diverse audience, including faculty, researchers, students, and professionals in Computation, AI, Data Science, and related fields. It serves as a forum for exchanging ideas, networking, and contributing to the growth of these rapidly evolving areas. We highly recommend in-person attendance to enhance engagement and networking opportunities with speakers and fellow participants.

Call for Participation: We welcome contributions from researchers, practitioners, and students. Whether presenting your work, participating in discussions, or attending as a learner, your involvement is crucial to the success of this collaborative initiative.