BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//https://events.wayne.edu///NONSGML kigkonsult.se iCalcreator 2.24
.2//
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:WSU - Research Events
X-WR-CALDESC:Wayne State University Events Calendar
X-WR-TIMEZONE:America/Detroit
X-LIC-LOCATION:America/Detroit
BEGIN:VEVENT
UID:20200919T160604EDT-39129185b2@https://events.wayne.edu/
DTSTAMP:20200919T200604Z
CATEGORIES:
CLASS:public
CREATED:20200210T091031Z
DESCRIPTION:Speaker: \;Professor Thaleia Zariphopoulou\, \;Presiden
tial Chair of Mathematics and the V. H. Neuhaus Centennial Professor of Fi
nance \;at the University of Texas at Austin\nTitle: \;Exploration
versus exploitation in reinforcement learning: a stochastic control appro
ach\nAbstract: \;In this talk\, I will consider reinforcement learning
(RL) in continuous time and study the problem of achieving the best trade
-off between exploration and exploitation. I will propose an entropy-regul
arized reward function involving the differential entropy of the distribut
ions of actions\, and motivate and devise an exploratory formulation for t
he feature dynamics that captures learning under exploration. The resultin
g optimization problem is a revitalization of the classical relaxed stocha
stic control. I will provide a complete analysis of the problem in the lin
ear&ndash\; quadratic (LQ) setting and deduce that the optimal feedback co
ntrol distribution for balancing exploitation and exploration is Gaussian.
This in turn interprets the widely adopted Gaussian exploration in RL\, b
eyond its simplicity for sampling. Moreover\, the exploitation and explora
tion are captured respectively by the mean and variance of the Gaussian di
stribution. Furthermore\, a more random environment contains more learning
opportunities in the sense that less exploration is needed. I will also c
haracterize the cost of exploration\, which\, for the LQ case\, is shown t
o be proportional to the entropy regularization weight and inversely propo
rtional to the discount factor. Finally\, as the weight of exploration dec
ays to zero\, I will discuss the convergence of the solution of the entrop
y-regularized LQ problem to the one of the classical LQ problem.
DTSTART:20200304T110000
DTEND:20200304T120000
LOCATION:State Hall
SUMMARY:Mathematics Colloquium - Thaleia Zariphopoulou
TRANSP:OPAQUE
URL:https://events.wayne.edu/research-events/2020/03/04/mathematics-colloqu
ium-thaleia-zariphopoulou-85413/
X-MICROSOFT-CDO-ALLDAYEVENT:false
END:VEVENT
END:VCALENDAR