Speaker: Koulik Khamaru, Department of Statistics, Rutgers University

Time: Wednesday, April 22, from 2:30 pm to 3:30 pm

Zoom link for online audience: https://wayne-edu.zoom.us/j/92845590121?pwd=CpRA5Wa5gzSMn2xiVkR2abD83O5nrH.1

Title: Bandit A/B testing via Stability: A Tale of Two Algorithms

Abstract: Modern decision-making increasingly relies on adaptive experimentation, particularly in settings such as A/B testing, multi-armed bandits, and reinforcement learning. While these methods enable more efficient learning and allocation of resources, they fundamentally challenge traditional statistical inference. Classical i.i.d.-based tools often break down under adaptive data collection, resulting in biased estimators and misleading confidence intervals.

This talk offers an overview of statistical inference in these adaptive environments through the concept of stability—originally formulated by Lai and Wei (1982).  A key advantage of stability is that it allows us to recover classical inferential guarantees—such as asymptotic normality and valid confidence intervals—even when the data arise from highly adaptive algorithms.

Next, we discuss the stability properties of two widely used algorithms: the Upper Confidence Bound (UCB) and Thompson Sampling. We argue that while UCB is stable, Thompson Sampling is not. Finally, we propose a modification of Thompson Sampling that regains stability while maintaining near-optimal regret. Key illustrations include quantitative central limit theorems of empirical mean in stochastic bandits and least square estimators in contextual bandits. We also present a new proof technique for analyzing regret and establishing stability, which we believe has broader applicability and may be ofindependent interest.

The talk is based on a series of joint works with Cunhui Zhang, Qiyang Han, Budhaditya Halder and Subhayan Pan.

Bio: Koulik Khamaru is an Assistant Professor in the Department of Statistics at Rutgers University. His research lies at the intersection of statistics and machine learning, with interests in Gaussian mixture models, convex and nonconvex optimization, and reinforcement learning. His recent work focuses on statistical inference for data collected through sequential reinforcement learning algorithms. Before joining Rutgers, he earned his PhD in Statistics from University of California Berkeley under the guidance of Prof. Martin  Wainwright and Prof. Michael Jordan.

______________________________________________________________________

AIDaS: CAD Seminar Series

Advancing Knowledge, Innovation, and Collaboration in Computation, AI, and Data Science (CAD)

The CAD Seminar Series is a primary seminar series at Wayne State University’s Institute for AI and Data Science (AIDaS). It is a dedicated platform for advancing knowledge, fostering innovation, and promoting collaboration across the fields of Computation, Artificial Intelligence, and Data Science. This series brings together leading experts, researchers, and professionals to explore the latest developments, tackle emerging challenges, and drive forward-thinking solutions at the convergence of these critical disciplines.

Objectives:

• Advance Knowledge: Share cutting-edge research and insights that push the boundaries of what is known in CAD.

• Foster Innovation: Encourage the development of novel ideas and solutions through interdisciplinary dialogue and creative thinking.

• Promote Collaboration: Unite expertise across disciplines and build bridges between academia, industry, and government to address complex problems and create opportunities for joint ventures.

Target Audience: The CAD Seminar Series is designed for a diverse audience, including faculty, researchers, students, and professionals in Computation, AI, Data Science, and related fields. It serves as a forum for exchanging ideas, networking, and contributing to the growth of these rapidly evolving areas. We highly recommend in-person attendance to enhance engagement and networking opportunities with speakers and fellow participants.

Call for Participation: We welcome contributions from researchers, practitioners, and students. Whether presenting your work, participating in discussions, or attending as a learner, your involvement is crucial to the success of this collaborative initiative.