Transient Response Analysis of Metropolis Learning in Games

Handle URI:
http://hdl.handle.net/10754/626610
Title:
Transient Response Analysis of Metropolis Learning in Games
Authors:
Jaleel, Hassan; Shamma, Jeff S. ( 0000-0001-5638-9551 )
Abstract:
The objective of this work is to provide a qualitative description of the transient properties of stochastic learning dynamics like adaptive play, log-linear learning, and Metropolis learning. The solution concept used in these learning dynamics for potential games is that of stochastic stability, which is based on the stationary distribution of the reversible Markov chain representing the learning process. However, time to converge to a stochastically stable state is exponential in the inverse of noise, which limits the use of stochastic stability as an effective solution concept for these dynamics. We propose a complete solution concept that qualitatively describes the state of the system at all times. The proposed concept is prevalent in control systems literature where a solution to a linear or a non-linear system has two parts, transient response and steady state response. Stochastic stability provides the steady state response of stochastic learning rules. In this work, we study its transient properties. Starting from an initial condition, we identify the subsets of the state space called cycles that have small hitting times and long exit times. Over the long time scales, we provide a description of how the distributions over joint action profiles transition from one cycle to another till it reaches the globally optimal state.
KAUST Department:
Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
Citation:
Jaleel H, Shamma JS (2017) Transient Response Analysis of Metropolis Learning in Games. IFAC-PapersOnLine 50: 9661–9667. Available: http://dx.doi.org/10.1016/j.ifacol.2017.08.1927.
Publisher:
Elsevier BV
Journal:
IFAC-PapersOnLine
Issue Date:
19-Oct-2017
DOI:
10.1016/j.ifacol.2017.08.1927
Type:
Article
ISSN:
2405-8963
Sponsors:
Research supported by funding from KAUST.
Additional Links:
http://www.sciencedirect.com/science/article/pii/S2405896317325569
Appears in Collections:
Articles; Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division

Full metadata record

DC FieldValue Language
dc.contributor.authorJaleel, Hassanen
dc.contributor.authorShamma, Jeff S.en
dc.date.accessioned2018-01-01T12:19:03Z-
dc.date.available2018-01-01T12:19:03Z-
dc.date.issued2017-10-19en
dc.identifier.citationJaleel H, Shamma JS (2017) Transient Response Analysis of Metropolis Learning in Games. IFAC-PapersOnLine 50: 9661–9667. Available: http://dx.doi.org/10.1016/j.ifacol.2017.08.1927.en
dc.identifier.issn2405-8963en
dc.identifier.doi10.1016/j.ifacol.2017.08.1927en
dc.identifier.urihttp://hdl.handle.net/10754/626610-
dc.description.abstractThe objective of this work is to provide a qualitative description of the transient properties of stochastic learning dynamics like adaptive play, log-linear learning, and Metropolis learning. The solution concept used in these learning dynamics for potential games is that of stochastic stability, which is based on the stationary distribution of the reversible Markov chain representing the learning process. However, time to converge to a stochastically stable state is exponential in the inverse of noise, which limits the use of stochastic stability as an effective solution concept for these dynamics. We propose a complete solution concept that qualitatively describes the state of the system at all times. The proposed concept is prevalent in control systems literature where a solution to a linear or a non-linear system has two parts, transient response and steady state response. Stochastic stability provides the steady state response of stochastic learning rules. In this work, we study its transient properties. Starting from an initial condition, we identify the subsets of the state space called cycles that have small hitting times and long exit times. Over the long time scales, we provide a description of how the distributions over joint action profiles transition from one cycle to another till it reaches the globally optimal state.en
dc.description.sponsorshipResearch supported by funding from KAUST.en
dc.publisherElsevier BVen
dc.relation.urlhttp://www.sciencedirect.com/science/article/pii/S2405896317325569en
dc.subjectgame theoryen
dc.subjectLearning theoryen
dc.subjectSensor networksen
dc.subjectStochastic controlen
dc.titleTransient Response Analysis of Metropolis Learning in Gamesen
dc.typeArticleen
dc.contributor.departmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Divisionen
dc.identifier.journalIFAC-PapersOnLineen
kaust.authorJaleel, Hassanen
kaust.authorShamma, Jeff S.en
All Items in KAUST are protected by copyright, with all rights reserved, unless otherwise indicated.