View source for Mean-field game theory

{{Short description|Study of strategic decision making}}
'''Mean-field game theory''' is the study of strategic decision making by small interacting [[agent (economics)|agents]] in very large populations. It lies at the intersection of [[game theory]] with stochastic analysis and [[control theory]]. The use of the term "mean field" is inspired by [[mean-field theory]] in physics, which considers the behavior of systems of large numbers of particles where individual particles have negligible impacts upon the system. In other words, each agent acts according to his minimization or maximization problem taking into account other agents’ decisions and because their population is large we can assume the number of agents goes to infinity and a representative agent exists.<ref>{{Cite arXiv |eprint=1907.01411 |first=Athanasios |last=Vasiliadis |title=An Introduction to Mean Field Games using probabilistic methods|year=2019 |class=math.OC }}</ref> 

In traditional [[game theory]], the subject of study is usually a game with two players and discrete time space, and extends the results to more complex situations by induction. However, for games in continuous time with continuous states (differential games or stochastic differential games) this strategy cannot be used because of the complexity that the dynamic interactions generate. On the other hand with MFGs we can handle large numbers of players through the mean representative agent and at the same time describe complex state dynamics.

This class of problems was considered in the economics literature by [[Boyan Jovanovic]] and [[Robert W. Rosenthal]],<ref>{{cite journal |last1=Jovanovic |first1=Boyan |last2=Rosenthal |first2=Robert W. |year=1988 |title=Anonymous Sequential Games |journal=[[Journal of Mathematical Economics]] |volume=17 |issue=1 |pages=77–87 |doi=10.1016/0304-4068(88)90029-8 }}</ref> in the engineering literature by Minyi Huang, Roland Malhame, and [[Peter E. Caines]]<ref>{{cite journal |first1=M. Y. |last1=Huang |first2=R. P. |last2=Malhame |first3=P. E. |last3=Caines |title=Large Population Stochastic Dynamic Games: Closed-Loop McKean–Vlasov Systems and the Nash Certainty Equivalence Principle |journal=Communications in Information and Systems |volume=6 |issue=3 |year=2006 |pages=221–252 |doi= 10.4310/CIS.2006.v6.n3.a5|zbl=1136.91349 |doi-access=free }}</ref><ref>{{cite journal |first1=M. |last1=Nourian |first2=P. E. |last2=Caines |title=ε–Nash mean field game theory for nonlinear stochastic dynamical systems with major and minor agents |journal=SIAM Journal on Control and Optimization |volume=51 |issue=4 |year=2013 |pages=3302–3331 |doi=10.1137/120889496 |arxiv=1209.5684 |s2cid=36197045 }}</ref><ref>{{cite journal |last1=Djehiche |first1=Boualem |last2=Tcheukam |first2=Alain |last3=Tembine |first3=Hamidou |title=Mean-Field-Type Games in Engineering |journal=AIMS Electronics and Electrical Engineering |date=2017 |volume=1 |issue=1 |pages=18–73 |doi=10.3934/ElectrEng.2017.1.18 |arxiv=1605.03281 |s2cid=16055840 }}</ref> and independently and around the same time by mathematicians {{ill|Jean-Michel Lasry|fr}} and [[Pierre-Louis Lions]].<ref>{{cite journal |last1=Lions |first1=Pierre-Louis |last2=Lasry |first2=Jean-Michel |title=Large investor trading impacts on volatility |journal=Annales de l'Institut Henri Poincaré C |date=March 2007 |volume=24 |issue=2 |pages=311–323 |doi=10.1016/j.anihpc.2005.12.006 |bibcode=2007AIHPC..24..311L |url=http://www.numdam.org/item/AIHPC_2007__24_2_311_0/ |doi-access=free }}</ref><ref>{{cite journal |last1=Lasry |first1=Jean-Michel |last2=Lions |first2=Pierre-Louis |title=Mean field games |journal=Japanese Journal of Mathematics |date=28 March 2007 |volume=2 |issue=1 |pages=229–260 |doi=10.1007/s11537-007-0657-8 |s2cid=1963678 |url=https://basepub.dauphine.fr/handle/123456789/2263 }}</ref>

In continuous time a mean-field game is typically composed of a [[Hamilton–Jacobi–Bellman equation]] that describes the [[optimal control]] problem of an individual and a [[Fokker–Planck equation]] that describes the dynamics of the aggregate distribution of agents. Under fairly general assumptions it can be proved that a class of mean-field games is the limit as <math>N \to \infty</math> of an ''N''-player [[Nash equilibrium]].<ref>{{Cite web |url=https://www.ceremade.dauphine.fr/~cardaliaguet/MFG20130420.pdf |title=Notes on Mean Field Games |last=Cardaliaguet |first=Pierre |date=September 27, 2013}}</ref>

A related concept to that of mean-field games is "mean-field-type control". In this case, a [[social planner]] controls the distribution of states and chooses a control strategy. The solution to a mean-field-type control problem can typically be expressed as a dual adjoint Hamilton–Jacobi–Bellman equation coupled with [[Fokker–Planck equation|Kolmogorov equation]]. Mean-field-type game theory is the multi-agent generalization of the single-agent mean-field-type control.<ref>{{Cite book |url=https://www.springer.com/gp/book/9781461485070 |title=Mean Field Games and Mean Field Type Control Theory |last1=Bensoussan |first1=Alain |last2=Frehse |first2=Jens |last3=Yam |first3=Phillip |date=2013 |publisher=Springer-Verlag |isbn=9781461485070 |series=Springer Briefs in Mathematics |location=New York |language=en}}{{page needed|date=May 2019}}</ref>

== General Form of a Mean-field Game ==
The following system of equations<ref>{{Cite book |last=Achdou |first=Yves |url=https://www.worldcat.org/oclc/1238206187 |title=Mean field games : Cetraro, Italy 2019 |date=2020 |others=Pierre Cardaliaguet, F. Delarue, Alessio Porretta, Filippo Santambrogio |isbn=978-3-030-59837-2 |location=Cham |oclc=1238206187}}</ref> can be used to model a typical Mean-field game:

<math>\begin{cases} -\partial_t u-\nu \Delta u+H(x,m,Du)=0 &(1)\\ \partial_t m-\nu \Delta m-\operatorname{div}(D_p H(x,m,Du) m)=0 &(2)\\ m(0)=m_0 &(3)\\ u(x,T)=G(x,m(T)) &(4) \end{cases}</math>

The basic dynamics of this set of Equations can be explained by an average agent's optimal control problem. In a mean-field game, an average agent can control their movement <math>\alpha</math> to influence the population's overall location by:

<math>d X_t=\alpha_t dt+\sqrt{2\nu} dB_t</math>

where <math>\nu</math> is a parameter and <math>B_t</math> is a standard Brownian motion. By controlling their movement, the agent aims to minimize their overall expected cost <math>C</math> throughout the time period <math>[0,T]</math>:

<math>C=\mathbb{E}\left[\int_{0}^TL(X_s,\alpha_s,m(s))ds+G(X_T,m(T))\right]</math>

where <math>L(X_s,\alpha_s,m(s))</math> is the running cost at time <math>s</math> and <math>G(X_T,m(T))
</math> is the terminal cost at time <math>T</math>. By this definition, at time <math>t</math> and position <math>x</math>, the value function <math>u(t,x)</math> can be determined as:

<math>u(t,x)=\inf_{\alpha}\mathbb{E}\left[\int_{t}^TL(X_s,\alpha_s,m(s))ds+G(X_T,m(T))\right]</math>

Given the definition of the value function <math>u(t,x)</math>, it can be tracked by the Hamilton-Jacobi equation (1). The optimal action of the average players <math>\alpha^*(x,t)</math> can be determined as <math>\alpha^*(x,t)=D_p H(x,m,Du)</math>. As all agents are relatively small and cannot single-handedly change the dynamics of the population, they will individually adapt the optimal control and the population would move in that way. This is similar to a Nash Equilibrium, in which all agents act in response to a specific set of others' strategies. The optimal control solution then leads to the Kolmogorov-Fokker-Planck equation (2).

== Finite State Games ==
A prominent category of mean field is games with a finite number of states and a finite number of actions per player. For those games, the analog of the Hamilton-Jacobi-Bellman equation is the Bellman equation, and the discrete version of the Fokker-Planck equation is the Kolmogorov equation. Specifically, for discrete-time models, the players' strategy is the Kolmogorov equation's probability matrix.  In continuous time models, players have the ability to control the transition rate matrix.

A discrete mean field game can be defined by a tuple <math>\mathcal{G}=(\mathcal{E}, \mathcal{A}, \{ Q_a \}, {\bf m }_0, \{ c_a \}, \beta)</math>, where <math>\mathcal{E}</math> is the state space, <math>\mathcal{A}</math> the action set, <math> Q_{a} </math> the transition rate matrices, <math>{\bf m }_0</math> the initial state, <math>\{c_a\}</math> the cost functions and <math>\beta</math> <math>\in \mathbb{R}</math> a discount factor. Furthermore, a mixed strategy is a measurable function <math>\pi: \mathbb{E} \times \mathbb{R}^+ \xrightarrow[]{} \mathcal{P(A)}</math>, that associates to each state <math>i \in \mathcal{E}</math> and each time <math>t \geq 0</math> a probability measure <math>\pi_i(t) \in \mathcal{P(A)}</math> on the set of possible actions. Thus <math>\pi_{i,a}(t)</math> is the probability that, at time <math>t</math> a player in state <math>i</math> takes action <math>a</math>, under strategy <math>\pi</math>. Additionally, rate matrices <math> \{ Q_a ({\bf m }^{\pi}(t)) \}_{a \in \mathcal{A}} </math> define the evolution over the time of population distribution, where <math>{\bf m }^{\pi}(t) \in \mathcal{P(\mathcal{E})}</math> is the population distribution at time <math>t</math>.<ref>{{cite journal | arxiv=1909.01209 | doi=10.3934/jdg.2019016 | title=Discrete mean field games: Existence of equilibria and convergence | year=2019 | last1=Doncel | first1=Josu | last2=Gast | first2=Nicolas | last3=Gaujal | first3=Bruno | journal=Journal of Dynamics & Games | pages=1–19 | s2cid=197507580 }}</ref>

== Linear-quadratic Gaussian game problem ==
From Caines (2009), a relatively simple model of large-scale games is the [[Linear–quadratic–Gaussian control|linear-quadratic Gaussian]] model. The individual agent's dynamics are modeled as a [[stochastic differential equation]]

<math display="block">dX_i = (a_i X_i + b_i u_i) \,dt + \sigma_i \,dW_i, \quad i = 1, \dots, N,</math>

where <math>X_i</math> is the state of the <math>i</math>-th agent, <math>u_i</math> is the control of the <math>i</math>-th agent, and <math>W_i</math> are independent [[Wiener process|Wiener processes]] for all <math>i = 1, \dots, N</math>. The individual agent's cost is

<math display="block">J_i(u_i, \nu) = \mathbb{E}\left\{ \int_0^\infty e^{-\rho t} \left[(X_i - \nu)^2 + ru_i^2\right] \,dt\right\}, \quad \nu = \Phi\left(\frac{1}{N} \sum_{k \neq i}^N X_k + \eta\right).</math>

The coupling between agents occurs in the cost function.

== General and Applied Use ==
The paradigm of Mean Field Games has become a major connection between distributed decision-making and stochastic modeling. Starting out in the stochastic control literature, it is gaining rapid adoption across a range of applications, including:

'''a. Financial market'''
Carmona reviews applications in financial engineering and economics that can be cast and tackled within the framework of the MFG paradigm.<ref>{{cite arXiv |last1=Carmona |first1=Rene |title=Applications of mean field games in financial engineering and economic theory |eprint=2012.05237 |date=2020|class=q-fin.GN }}</ref> Carmona argues that models in macroeconomics, contract theory, finance, …, greatly benefit from the switch to continuous time from the more traditional discrete-time models. He considers only continuous time models in his review chapter, including systemic risk, price impact, optimal execution, models for bank runs, high-frequency trading, and cryptocurrencies.

'''b. Crowd motions'''
MFG assumes that individuals are smart players which try to optimize their strategy and path with respect to certain costs (equilibrium with rational expectations approach). MFG models are useful to describe the anticipation phenomenon: the forward part describes the crowd evolution while the backward gives the process of how the anticipations are built. Additionally, compared to multi-agent microscopic model computations, MFG only requires lower computational costs for the macroscopic simulations. Some researchers have turned to MFG in order to model the interaction between populations and study the decision-making process of intelligent agents, including aversion and congestion behavior between two groups of pedestrians,<ref>{{cite journal |last1=Lachapelle |first1=Aimé |last2=Wolfram |first2=Marie-Therese|author2-link=Marie-Therese Wolfram |title=On a mean field game approach modeling congestion and aversion in pedestrian crowds |journal=Transportation Research Part B: Methodological |date=2011 |volume=45 |issue=10 |pages=1572–1589|doi=10.1016/j.trb.2011.07.011 |s2cid=55991774 |url=https://basepub.dauphine.fr/handle/123456789/5946 }}</ref> departure time choice of morning commuters,<ref>{{cite arXiv |last1=Feinstein |first1=Zachary |last2=Sojmark |first2=Andreas |title=A dynamic default contagion model: From Eisenberg-Noe to the mean field |eprint=1912.08695 |date=2019|class=q-fin.MF }}</ref> and decision-making processes for autonomous vehicle.<ref>{{cite journal |last1=Huang |first1=Kuang |last2=Chen |first2=Xu |last3=Di |first3=Xuan |last4=Du |first4=Qiang |title=Dynamic driving and routing games for autonomous vehicles on networks: A mean field game approach |journal=Transportation Research Part C: Emerging Technologies |date=2021 |volume=128 |page=103189|doi=10.1016/j.trc.2021.103189 |s2cid=235436377 |arxiv=2012.08388 }}</ref> 

'''c. Control and mitigation of Epidemics'''
Since the epidemic has affected society and individuals significantly, MFG and mean-field controls (MFCs) provide a perspective to study and understand the underlying population dynamics, especially in the context of the Covid-19 pandemic response. MFG has been used to extend the SIR-type dynamics with spatial effects or allowing for individuals to choose their behaviors and control their contributions to the spread of the disease. MFC is applied to design the optimal strategy to control the virus spreading within a spatial domain,<ref>{{cite journal |last1=Lee |first1=Wonjun |last2=Liu |first2=Siting |last3=Tembine |first3=Hamidou |last4=Li |first4=Wuchen |last5=Osher |first5=Stanley |title=Controlling propagation of epidemics via mean-field control |journal=SIAM Journal on Applied Mathematics |date=2021 |volume=81 |issue=1 |pages=190–207|doi=10.1137/20M1342690 |arxiv=2006.01249 |s2cid=226299517 }}</ref> control individuals’ decisions to limit their social interactions,<ref>{{cite journal |last1=Aurell |first1=Alexander |last2=Carmona |first2=Rene |last3=Dayanikli |first3=Gokce |last4=Lauriere |first4=Mathieu |title=Optimal incentives to mitigate epidemics: a Stackelberg mean field game approach |journal=SIAM Journal on Control and Optimization |date=2022 |volume=60 |issue=2 |page=S294–S322|doi=10.1137/20M1377862 |arxiv=2011.03105 |s2cid=226278147 }}</ref> and support the government’s nonpharmaceutical interventions.<ref>{{cite journal |last1=Elie |first1=Romuald |last2=Hubert |first2=Emma |last3=Turinici |first3=Gabriel |title=Contact rate epidemic control of COVID-19: an equilibrium view |journal=Mathematical Modelling of Natural Phenomena |date=2020 |volume=15 |page=35|doi=10.1051/mmnp/2020022 |s2cid=215814201 |doi-access=free |arxiv=2004.08221 }}</ref>

==See also==
{{div col|colwidth=25em}}
* [[Aggregative game]]
* [[Complex adaptive system]]
* [[Differential game]]
* [[Evolutionary game theory]]
* [[Quantal response equilibrium]]
* [[Potential game#Bounded Rational Models|Potential game]]
{{div col end}}

==References==
{{Reflist|30em}}

==External links==
* {{usurped|1=[https://web.archive.org/web/20120415020409/http://www.ieeecss-oll.org/lectures/2009/mean-field-stochastic-control Mean Field Stochastic Control]}} ({{usurped|1=[https://web.archive.org/web/20150924032856/http://www.ieeecss-oll.org/sites/default/files/Caines.pdf Slides]}}), 2009 IEEE Control Systems Society Bode Prize Lecture by [[Peter E. Caines]]
*{{cite book |doi=10.1007/978-1-4471-5102-9_30-1 |chapter=Mean Field Games |title=Encyclopedia of Systems and Control |pages=1–6 |year=2013 |last1=Caines |first1=Peter E. |isbn=978-1-4471-5102-9 |s2cid=33954904 }}
* [https://www.ceremade.dauphine.fr/~cardaliaguet/MFG20130420.pdf Notes on Mean Field Games], from [[Pierre-Louis Lions]]' lectures at [[Collège de France]]
* {{in lang|fr}} [http://www.college-de-france.fr/site/pierre-louis-lions/index.htm#|m=course|q=/site/pierre-louis-lions/course-2011-2012.htm Video lectures] by Pierre-Louis Lions
* [https://www.oliviergueant.com/uploads/4/3/0/9/4309511/paris-princeton.pdf Mean field games and applications] by Olivier Guéant, Jean-Michel Lasry, and Pierre-Louis Lions

{{Game theory}}

[[Category:Game theory]]
[[Category:Mathematical economics]]