View source for Evolutionary game theory

{{Short description|Application of game theory to evolving populations in biology}}
'''Evolutionary game theory''' ('''EGT''') is the application of [[game theory]] to evolving populations in [[biology]]. It defines a framework of contests, strategies, and analytics into which [[Darwinism|Darwinian]] competition can be modelled. It originated in 1973 with [[John Maynard Smith]] and [[George R. Price]]'s formalisation of contests, analysed as strategies, and the mathematical criteria that can be used to predict the results of competing strategies.<ref name=Price>{{Cite journal |last1=Maynard Smith |first1=John | last2=Price |first2=G. R. | title=The Logic of Animal Conflict | journal=Nature | year=1973 | volume=246 | issue=5427 | pages=15–18 | s2cid=4224989| doi=10.1038/246015a0|bibcode=1973Natur.246...15S}}</ref>

Evolutionary game theory differs from classical game theory in focusing more on the dynamics of strategy change.<ref name=Newton2018renaissance>{{cite journal | last1=Newton | first1=Jonathan | year=2018 | title=Evolutionary Game Theory: A Renaissance | journal=Games | volume=9 | issue=2| pages=31 | doi=10.3390/g9020031| url=https://www.econstor.eu/bitstream/10419/179191/1/games-09-00031-v2.pdf | doi-access=free }}</ref> This is influenced by the frequency of the competing strategies in the population.<ref>{{Cite book |title=Networks, Crowds, and Markets: Reasoning About a Highly Connected World |last1=Easley |first1=David |last2=Kleinberg |first2=Jon |publisher=Cambridge University Press |year=2010 |isbn=9780521195331 |url=http://www.cs.cornell.edu/home/kleinber/networks-book/networks-book-ch07.pdf}}</ref>

Evolutionary game theory has helped to explain the basis of [[altruism (biology)|altruistic]] behaviours in Darwinian [[evolution]]. It has in turn become of interest to [[economists]],<ref>{{cite book |first=Kandori |last=Michihiro |authorlink=Michihiro Kandori |chapter=Evolutionary game theory in economics |pages=243–277 |title=Advances in Economics and Econometrics : Theory and Applications |volume=1 |editor-first=David M. |editor-last=Kreps |editor2-first=Kenneth F. |editor2-last=Wallis |location= |publisher=Cambridge University Press |year=1997 |isbn=0-521-58983-5 }}</ref> [[sociologists]], [[anthropologists]], and [[philosophers]].

==History==
===Classical game theory===
{{Main|Game theory}}

Classical [[non-cooperative game theory]] was conceived by [[John von Neumann]] to determine optimal strategies in competitions between adversaries. A contest involves players, all of whom have a choice of moves. Games can be a single round or repetitive. The approach a player takes in making their moves constitutes their strategy. Rules govern the outcome for the moves taken by the players, and outcomes produce payoffs for the players; rules and resulting payoffs can be expressed as [[decision tree]]s or in a [[Matrix (mathematics)|payoff matrix]]. Classical theory requires the players to make rational choices. Each player must consider the strategic analysis that their opponents are making to make their own choice of moves.<ref>{{citation |first=J. v. |last=Neumann |s2cid=122961988 |year=1928 |title=Zur Theorie der Gesellschaftsspiele |journal=[[Mathematische Annalen]] |volume=100 |issue=1 |pages=295–320 |doi=10.1007/BF01448847}} English translation: {{citation |chapter=On the Theory of Games of Strategy |editor1-first=A. W. |editor1-last=Tucker |editor2-first=R. D. |editor2-last=Luce |year=1959 |title=Contributions to the Theory of Games |volume=4 |pages=13–42 |publisher=Princeton University Press |isbn=0691079374 |chapter-url=https://books.google.com/books?id=9lSVFzsTGWsC&pg=PA13}}</ref><ref>{{cite book |first=Philip |last=Mirowski |chapter=What Were von Neumann and Morgenstern Trying to Accomplish? |editor-first=E. Roy |editor-last=Weintraub |title=Toward a History of Game Theory |location=Durham |publisher=Duke University Press |year=1992 |isbn=978-0-8223-1253-6 |pages=113–147 |chapter-url=https://books.google.com/books?id=9CHY2Gozh1MC&pg=PA113 }}</ref>

===The problem of ritualized behaviour===
[[File:John Maynard Smith.jpg|thumb|The mathematical biologist [[John Maynard Smith]] modelled evolutionary games.]]

Evolutionary game theory started with the problem of how to explain ritualized animal behaviour in a conflict situation; "why are animals so 'gentlemanly or ladylike' in contests for resources?" The leading [[ethologists]] [[Niko Tinbergen]] and [[Konrad Lorenz]] proposed that such behaviour exists [[group selection|for the benefit of the species]]. [[John Maynard Smith]] considered that incompatible with Darwinian thought,<ref>{{cite book |author=Cohen, Marek |title=A Reason for Everything |date=2004 |publisher=Faber and Faber |pages=231–240 |isbn=978-0-571-22393-0}}</ref> where selection occurs at an individual level, so self-interest is rewarded while seeking the common good is not. Maynard Smith, a mathematical biologist, turned to game theory as suggested by George Price, though [[Richard Lewontin]]'s attempts to use the theory had failed.<ref>Video Interview – John Maynard Smith – [http://www.webofstories.com/play/7294?o=MS The creation of Evolutionary Game Theory]</ref>

===Adapting game theory to evolutionary games===

Maynard Smith realised that an evolutionary version of game theory does not require players to act rationally—only that they have a strategy. The results of a game show how good that strategy was, just as [[evolution]] tests alternative strategies for the ability to survive and reproduce. In biology, strategies are genetically inherited traits that control an individual's action, analogous with computer programs.  The success of a strategy is determined by how good the strategy is in the presence of competing strategies (including itself), and of the frequency with which those strategies are used.<ref>{{cite book |author=Vincent, Thomas |title=Evolutionary Game Theory, Natural Selection, and Darwinian Dynamics |url=https://archive.org/details/evolutionarygame00tlvi |url-access=limited |date=2005 |publisher=Cambridge University Press |pages=[https://archive.org/details/evolutionarygame00tlvi/page/n90 72]–87 |isbn=978-0-521-84170-2}}</ref> Maynard Smith described his work in his book ''[[Evolution and the Theory of Games]]''.<ref name=JMSbook>{{cite book |author-link=John Maynard Smith |last=Maynard Smith |first=John |date=1982 |title=Evolution and the Theory of Games |isbn=978-0-521-28884-2|title-link=Evolution and the Theory of Games |publisher=Cambridge University Press }}</ref>

Participants aim to produce as many replicas of themselves as they can, and the payoff is in units of fitness (relative worth in being able to reproduce). It is always a multi-player game with many competitors.  Rules include replicator dynamics, in other words how the fitter players will spawn more replicas of themselves into the population and how the less fit will be [[Culling|culled]], in a [[replicator equation]]. The replicator dynamics models heredity but not mutation, and assumes asexual reproduction for the sake of simplicity. Games are run repetitively with no terminating conditions. Results include the dynamics of changes in the population, the success of strategies, and any equilibrium states reached. Unlike in classical game theory, players do not choose their strategy and cannot change it: they are born with a strategy and their offspring inherit that same strategy.<ref>{{cite book |author=Dugatkin, Lee |title=Game Theory and Animal Behavior |date=1998  |publisher=Oxford University Press |isbn=978-0-19-509692-7 |pages=2–20}}</ref>

==Evolutionary games==
===Models===
[[File:Game Diagram AniFin.gif|thumb|400px|Evolutionary game theory analyses Darwinian mechanisms with a [[system model]] with three main components –  ''population'', ''game'', and ''replicator dynamics''. The system process has four phases:<br />
     <br />
1) The model (as evolution itself) deals with a ''population'' (Pn).  The population will exhibit [[Evolution#Sources of variation|variation]] among competing individuals.  In the model this competition is represented by the game.<br>
     <br>
2) The game tests the strategies of the individuals under the rules of the game.  These rules produce different payoffs – in units of [[Fitness (biology)|fitness]] (the production rate of offspring).  The contesting individuals meet in pairwise contests with others, normally in a highly mixed distribution of the population.  The mix of strategies in the population affects the payoff results by altering the odds that any individual may meet up in contests with various strategies. The individuals leave the game pairwise contest with a resulting fitness determined by the contest outcome, represented in a ''payoff matrix''.<br> 
<br>
3) Based on this resulting fitness each member of the population then undergoes replication or culling determined by the exact mathematics of the ''replicator dynamics process''.  This overall process then produces a ''new generation'' P(n+1).  Each surviving individual now has a new fitness level determined by the game result.<br>
<br>
4) The new generation then takes the place of the previous one and the cycle repeats. The population mix may converge to an ''evolutionarily stable state'' that cannot be invaded by any mutant strategy.]]

Evolutionary game theory encompasses Darwinian evolution, including competition (the game), natural selection (replicator dynamics), and heredity. Evolutionary game theory has contributed to the understanding of [[group selection]], [[sexual selection]], [[altruism]], [[parental care]], [[co-evolution]], and [[ecology|ecological]] dynamics. Many counter-intuitive situations in these areas have been put on a firm mathematical footing by the use of these models.<ref name=Hammerstein>{{cite book |author1=Hammerstein, Peter |author2-link=Reinhard Selten |author2=Selten, Reinhard |chapter=Game theory and evolutionary biology |title=Handbook of Game Theory with Economic Applications |volume=2 |editor1=Aumann, R.|editor2=Hart, S. |publisher=Elsevier |date=1994 |pages=929–993 |doi=10.1016/S1574-0005(05)80060-8 |isbn=978-0-444-89427-4}}</ref>

The common way to study the evolutionary dynamics in games is through [[replicator equation]]s. These show the growth rate of the proportion of organisms using a certain strategy and  that rate is equal to the difference between the average payoff of that strategy and the average payoff of the population as a whole.<ref name=Samuelson>{{cite journal | last1=Samuelson | first1=L. | year=2002 | title=Evolution and game theory | journal=  Journal of Economic Perspectives| volume=16 | issue=2| pages=46–66 | doi=10.1257/0895330027256 | doi-access=free }}</ref> Continuous replicator equations assume infinite populations, [[continuous time]], [[complete mixing]] and that strategies breed true. Some [[attractor]]s (all global asymptotically stable fixed points) of the equations are [[evolutionarily stable state]]s.<ref name="Zeeman">{{Citation
| last1 = Zeeman | first1 = E. C.
| contribution = Population dynamics from game theory
| title = Global theory of dynamical systems
| publisher = Springer Verlag
| place = Berlin
| pages = 471–497
| date = 1980
|contribution-url = http://www.lms.ac.uk/sites/lms.ac.uk/files/1980%20Population%20dynamics%20from%20game%20theory%20(preprint).pdf}}</ref>  A strategy which can survive all "mutant" strategies is considered evolutionarily stable. In the context of animal behavior, this usually means such strategies are programmed and heavily influenced by [[genetics]], thus making any player or organism's strategy determined by these biological factors.<ref>{{cite book |author=Weibull, J. W. |date=1995 |title=Evolutionary game theory |publisher=MIT Press }}</ref><ref>{{cite book |author1=Hofbauer, J. |author2-link=Karl Sigmund |author2=Sigmund, K.  |date=1998 |title=Evolutionary games and population dynamics |publisher=Cambridge University Press }}</ref>

Evolutionary games are mathematical objects with different rules, payoffs, and mathematical behaviours. Each "game" represents different problems that organisms have to deal with, and the strategies they might adopt to survive and reproduce. Evolutionary games are often given colourful names and cover stories which describe the general situation of a particular game. Representative games include [[Chicken (game)|hawk-dove]],<ref name=Price/> [[war of attrition (game)|war of attrition]],<ref name=SelfishGene/> [[stag hunt]], [[Cheating (biology)|producer-scrounger]], [[tragedy of the commons]], and [[prisoner's dilemma]]. Strategies for these games include hawk, dove, bourgeois, prober, defector, assessor, and retaliator. The various strategies compete under the particular game's rules, and the mathematics are used to determine the results and behaviours.

===Hawk dove===
[[File:HawkDove2.jpg|thumb|300px| Solution of the [[Chicken (game)|hawk dove]] game for V=2, C=10 and fitness starting base B=4. The fitness of a hawk for different population mixes is plotted as a black line, that of dove in red. An ESS (a stationary point) will exist when hawk and dove fitness are equal: Hawks are 20% of population and doves are 80% of the population.]]
{{main|Chicken (game)}}

The first game that [[Maynard Smith]] analysed is the classic ''[[Chicken (game)|hawk dove]]''{{efn|Maynard Smith chose the name "hawk dove" from descriptions of political views current during the [[Vietnam War]].}} game. It was conceived to analyse Lorenz and Tinbergen's problem, a contest over a shareable resource. The contestants can be either  a hawk or a dove. These are two subtypes or morphs of one species with different strategies. The hawk first displays aggression, then escalates into a fight until it either wins or is injured (loses). The dove first displays aggression, but if faced with major escalation runs for safety. If not faced with such escalation, the dove attempts to share the resource.<ref name=Price/>

{| class="wikitable" style="text-align:center"
|+style="white-space:nowrap"| Payoff matrix for hawk dove game
|-
|  || ''' meets hawk ''' || ''' meets dove '''
|-
| '''if hawk''' ||V/2 − C/2||V
|-
|'''if dove''' ||0 ||V/2
|}

Given that the resource is given the value V, the damage from losing a fight is given cost C:<ref name=Price/>

*If a hawk meets a dove, the hawk gets the full resource V 
*If a hawk meets a hawk, half the time they win, half the time they lose, so the average outcome is then V/2 minus C/2
*If a dove meets a hawk, the dove will back off and get nothing – 0
*If a dove meets a dove, both share the resource and get V/2

The actual payoff, however, depends on the probability of meeting a hawk or dove, which in turn is a representation of the percentage of hawks and doves in the population when a particular contest takes place. That, in turn, is determined by the results of all of the previous contests. If the cost of losing C is greater than the value of winning V (the normal situation in the natural world) the mathematics ends in an [[evolutionarily stable strategy]] (ESS), a mix of the two strategies where the population of hawks is V/C. The population regresses to this equilibrium point if any new hawks or doves make a temporary perturbation in the population.
The solution of the hawk dove game explains why most animal contests involve only [[Ritualized aggression|ritual fighting]] behaviours in contests rather than outright battles. The result does not at all depend on "[[group selection|good of the species]]" behaviours as suggested by Lorenz, but solely on the implication of actions of so-called [[selfish genes]].<ref name=Price/>

===War of attrition===
{{main|War of attrition (game)}}

In the hawk dove game the resource is shareable, which gives payoffs to both doves meeting in a pairwise contest. Where the resource is not shareable, but an alternative resource might be available by backing off and trying elsewhere, pure hawk or dove strategies are less effective. If an unshareable resource is combined with a high cost of losing a contest (injury or possible death) both hawk and dove payoffs are further diminished.  A safer strategy of lower cost display, bluffing and waiting to win, is then viable – a bluffer strategy. The game then becomes one of accumulating costs, either the costs of displaying or the costs of prolonged unresolved engagement. It is effectively an auction; the winner is the contestant who will swallow the greater cost while the loser gets the same cost as the winner but no resource.<ref name=SelfishGene>{{harvp|Dawkins|2006|pp=76-78}}</ref> The resulting evolutionary game theory mathematics lead to an optimal strategy of timed bluffing.<ref>{{cite book |author-link=John Maynard Smith |last=Maynard Smith |first=John |date=1982 |title=Evolution and the Theory of Games |isbn=978-0-521-28884-2 |page=[https://archive.org/details/evolutiontheoryg00jmsm/page/n35 28]|title-link=Evolution and the Theory of Games |publisher=Cambridge University Press }}</ref>

[[File:Attrition graph.jpg|thumb|300px| [[War of attrition (game)|War of attrition]] for different values of resource. Note the time it takes for an accumulation of 50% of the contestants to quit vs. the value (V) of resource contested for.]]

This is because in the war of attrition any strategy that is unwavering and predictable is unstable, because it will ultimately be displaced by a mutant strategy which relies on the fact that it can best the existing predictable strategy by investing an extra small delta of waiting resource to ensure that it wins. Therefore, only a random unpredictable strategy can maintain itself in a population of bluffers. The contestants in effect choose an acceptable cost to be incurred related to the value of the resource being sought, effectively making a random bid as part of a mixed strategy (a strategy where a contestant has several, or even many, possible actions in their strategy). This implements a distribution of bids for a resource of specific value V, where the bid for any specific contest is chosen at random from that distribution.  The distribution (an ESS) can be computed using the [[Bishop-Cannings theorem]], which holds true for any mixed-strategy ESS.<ref>{{cite book |author-link=John Maynard Smith |last=Maynard Smith |first=John |date=1982 |title=Evolution and the Theory of Games |isbn=978-0-521-28884-2 |page=[https://archive.org/details/evolutiontheoryg00jmsm/page/n40 33]|title-link=Evolution and the Theory of Games |publisher=Cambridge University Press }}</ref> The distribution function in these contests was determined by Parker and Thompson to be:

:<math>p(x)=\frac{e^{-x/V}}{V}.</math>
 
The result is that the cumulative population of quitters for any particular cost m in this "mixed strategy" solution is:

:<math>p(m)=1- e^{-m/V},</math>
   
as shown in the adjacent graph.  The intuitive sense that greater values of resource sought leads to greater waiting times is borne out. This is observed in nature, as in male dung flies contesting for mating sites, where the timing of disengagement in contests is as predicted by evolutionary theory mathematics.<ref>{{cite journal | last1=Parker | last2=Thompson | year=1980 | title=Dung Fly Struggle: a test of the War of Attrition | journal=Behavioral Ecology and Sociobiology | volume=7 | issue=1 | pages=37–44 | doi=10.1007/bf00302516| s2cid=44287633 }}</ref>

===Asymmetries that allow new strategies===

{{multiple image
 | width=200
 | footer=Animal strategy examples: by examining the behaviours, then determining both the costs and the values of resources attained in a contest the strategy of an organism can be verified 
 | image1=Scatophaga stercoraria.jpg
 | caption1= Dung fly (''Scatophaga stercoraria'') – a war of attrition player
 | image2=30-EastTimor-Dive2 Maubara 35 (Mantis Shrimp)-APiazza.JPG
 | caption2=The mantis shrimp guarding its home with the bourgeois strategy
 }}

In the war of attrition there must be nothing that signals the size of a bid to an opponent, otherwise the opponent can use the cue in an effective counter-strategy.  There is however a mutant strategy which can better a bluffer in the [[War of attrition (game)|war of attrition]] game if a suitable asymmetry exists, the bourgeois strategy. Bourgeois uses an asymmetry of some sort to break the deadlock. In nature one such asymmetry is possession of a resource. The strategy is to play a hawk if in possession of the resource, but to display then retreat if not in possession.  This requires greater cognitive capability than hawk, but bourgeois is common in many animal contests, such as in contests among [[mantis shrimp]]s and among [[speckled wood butterfly|speckled wood butterflies]].

===Social behaviour===
[[File:Game Theory Strategic Social Alternatives.jpg|thumb|300px|Alternatives for game theoretic social interaction]]

Games like hawk dove and war of attrition represent pure competition between individuals and have no attendant social elements.  Where social influences apply, competitors have four possible alternatives for strategic interaction.  This is shown on the adjacent figure, where a plus sign represents a benefit and a minus sign represents a cost.

* In a ''cooperative'' or ''mutualistic'' relationship both "donor" and "recipient" are almost indistinguishable as both gain a benefit in the game by co-operating, i.e. the pair are in a game-wise situation where both can gain by executing a certain strategy,  or alternatively both must act in  concert because of some encompassing  constraints that effectively puts them  "in the same boat".
* In an ''altruistic'' relationship the donor, at a cost to themself provides a benefit to the recipient.  In the general case the recipient will have a kin relationship to the donor and the donation is one-way.  Behaviours where benefits are donated alternatively (in both directions) at a cost, are often called "altruistic", but on analysis such "altruism" can be seen to arise from optimised  "selfish" strategies.
* ''Spite'' is essentially a “reversed” form of cooperation where neither party receives a tangible benefit. The general case is that the ally is kin related and the benefit is an easier competitive environment for the ally.   <small>Note: George Price, one of the early mathematical modellers of both altruism and spite, found this equivalence particularly disturbing at an emotional level.</small><ref>{{cite book |author=Harman, O. |title=The Price of Altruism |date=2010 |publisher=Bodley Head |pages=Chapter 9 |isbn=978-1-847-92062-1}}</ref>
* ''Selfishness'' is the base criteria of all strategic choice from a game theory perspective – strategies not aimed at self-survival and self-replication are not long for any game. Critically however, this situation is impacted by the fact that competition is taking place on multiple levels  – i.e. at a genetic, an individual and a group level.

==Contests of selfish genes==
[[File:Spermophilus beldingi.jpg|thumb|upright|Female [[Belding's ground squirrel]]s risk their lives giving loud alarm calls, protecting closely related female colony members; males are less closely related and do not call.<ref>{{cite book |author=Dugatkin, Alan |title=Principles of Animal Behavior |date=2004 |publisher=WW Norton |pages=255–260 |isbn=978-0-393-97659-5}}</ref>]]

At first glance it may appear that the contestants of evolutionary games are the individuals present in each generation who directly participate in the game. But individuals live only through one game cycle, and instead it is the strategies that really contest with one another over the duration of these many-generation games.  So it is ultimately genes that play out a full contest – selfish genes of strategy.  The contesting genes are present in an individual and to a degree in all of the individual's kin.  This can sometimes profoundly affect which strategies survive, especially with issues of cooperation and defection. [[W. D. Hamilton|William Hamilton]],<ref>Sigmund, Karl, Institute of Mathematics University of Vienna,  "William D. Hamilton’s Work in Evolutionary Game Theory", Interim Report IR-02-019</ref> known for his theory of [[kin selection]], explored many of these cases using game-theoretic models. Kin-related treatment of game contests<ref name=Brembs>{{cite book |last=Brembs |first=B. |year=2001 |doi=10.1006/rwgn.2001.0581 |chapter=Hamilton's Theory |title=Encyclopedia of Genetics |pages=906–910 |publisher=Academic Press|isbn=978-0-12-227080-2|url=https://epub.uni-regensburg.de/28603/1/brembs.pdf }}</ref> helps to explain many aspects of the behaviour of [[social insects]], the altruistic behaviour in parent-offspring interactions, mutual protection behaviours, and co-operative [[parental care|care of offspring]]. For such games, Hamilton defined an extended form of fitness – ''[[inclusive fitness]]'', which includes an individual's offspring as well as any offspring equivalents found in kin.
 
{| class="wikitable"
|-
! The mathematics of kin selection
|-
| The concept of ''kin selection'' is that:
:''inclusive fitness=own contribution to fitness + contribution of all relatives''.

Fitness is measured relative to the average population; for example, fitness=1 means growth at the average rate for the population, fitness < 1 means having a decreasing share in the population (dying out), fitness > 1 means an increasing share in the population (taking over).

The inclusive fitness of an individual '''w<sub>i</sub>''' is the sum of its specific fitness of itself '''a<sub>i</sub>''' plus the specific fitness of each and every relative weighted by the degree of relatedness which equates to the ''summation'' of all '''r<sub>j</sub>*b<sub>j</sub>'''....... where '''r<sub>j</sub>''' is relatedness of a specific relative and '''b<sub>j</sub>''' is that specific relative's fitness – producing:

:<math>w_i=a_i+\sum_{j}r_jb_j. </math>

If individual a<sub>i</sub> sacrifices their "own average equivalent fitness of 1" by accepting a fitness cost C, and then to "get that loss back", w<sub>i</sub> must still be 1 (or greater than 1)...and using '''R*B''' to represent the summation results in:

:<big>'''''1< (1-C)+RB'''''</big>  ....or rearranging.....    <big>'''''R>C/B'''''.</big><ref name=Brembs/>
|}
Hamilton went beyond kin relatedness to work with [[Robert Axelrod (political scientist)|Robert Axelrod]], analysing games of co-operation under conditions not involving kin where [[reciprocal altruism]] came into play.<ref>{{cite journal |author1=Axelrod, R. |author2=Hamilton, W.D. |author2-link=W. D. Hamilton |date=1981 |title=The evolution of cooperation |journal=Science |volume=211 |issue=4489 |pages=1390–1396 |doi=10.1126/science.7466396 |pmid=7466396|bibcode=1981Sci...211.1390A }}</ref>

===Eusociality and kin selection===
[[File:Meat eater ant feeding on honey.jpg|thumb|[[Iridomyrmex purpureus|Meat ant]] workers (always female) are related to a parent by a factor of 0.5, to a sister by 0.75, to a child by 0.5 and to a brother by 0.25.
Therefore, it is significantly more advantageous to help produce a sister (0.75) than to have a child (0.5).]]
{{main|Eusociality}}

[[Eusocial]] insect workers forfeit reproductive rights to their queen. It has been suggested that kin selection, based on the genetic makeup of these workers, may predispose them to altruistic behaviours.<ref>{{cite journal | last1=Hughes | last2=Oldroyd | last3=Beekman | last4=Ratnieks | year=2008 | title=Ancestral Monogamy Shows Kin Selection Is Key to the Evolution of Eusociality | journal=Science | volume=320 | issue=5880| pages=1213–1216 | doi=10.1126/science.1156108 | pmid=18511689 |bibcode=2008Sci...320.1213H | s2cid=20388889 }}</ref> Most eusocial insect societies have [[haplodiploid]] sexual determination, which means that workers are unusually closely related.<ref>{{cite journal | last1=Thorne | first1=B. | year=1997 | title=Evolution of Eusociality in Termites | journal=Annual Review of Ecology and Systematics | volume=28 | issue=1| pages=27–54 | doi=10.1146/annurev.ecolsys.28.1.27 | pmc=349550 }}</ref>

This explanation of insect eusociality has, however, been challenged by a few highly-noted evolutionary game theorists (Nowak and Wilson)<ref>{{cite journal | last1=Nowak | first1=Tarnita | last2=Wilson | year=2010| title=The evolution of eusociality | journal=Nature | volume=466 | issue=7310| pages=1057–1062 | doi=10.1038/nature09205 | pmid=20740005 | pmc=3279739|bibcode=2010Natur.466.1057N }}</ref> who have published a controversial alternative game theoretic explanation based on a sequential development and group selection effects proposed for these insect species.<ref>{{cite journal | last1=Bourke | first1=Andrew | year=2011| title=The validity and value of inclusive fitness theory | journal= Proceedings of the Royal Society B: Biological Sciences| volume=278 | issue=1723| pages=3313–3320 | doi=10.1098/rspb.2011.1465| pmid=21920980 | pmc=3177639}}</ref>

===Prisoner's dilemma===
{{main|Prisoner's dilemma}}

A difficulty of the theory of evolution, recognised by Darwin himself, was the problem of [[altruism]]. If the basis for selection is at an individual level, altruism makes no sense at all. But universal selection at the group level (for the good of the species, not the individual) fails to pass the test of the mathematics of game theory and is certainly not the general case in nature.<ref>{{cite book |author=Okasha, Samir |title=Evolution and the Levels of Selection |date=2006 |publisher=Oxford University Press |isbn=978-0-19-926797-2 }}</ref> Yet in many social animals, altruistic behaviour exists. The solution to this problem can be found in the application of evolutionary game theory to the [[prisoner's dilemma]] game – a game which tests the payoffs of cooperating or in defecting from cooperation.  It is the most studied game in all of game theory.<ref>{{cite journal |author1=Pacheco, Jorge M. |author2=Santos, Francisco C. |author3=Souza, Max O. |author4=Skyrms, Brian  |title=Evolutionary dynamics of collective action in N-person stag hunt dilemmas |journal=Proceedings of the Royal Society |date=2009 |doi=10.1098/rspb.2008.1126 |pmid=18812288 |volume=276 |issue=1655 |pages=315–321|pmc=2674356 }}</ref>

The analysis of the prisoner's dilemma is as a repetitive game. This affords competitors the possibility of retaliating for defection in previous rounds of the game. Many strategies have been tested; the best competitive strategies are general cooperation, with a reserved retaliatory response if necessary.<ref>{{cite book |author=Axelrod, R. |date=1984 |title=The Evolution of Cooperation|publisher=Basic Books |isbn=0-465-02121-2|edition=1st}} {{cite book |author=Axelrod, R. |date=2009 |title=The Evolution of Cooperation|edition=Revised |isbn=<!--0-465-02121-2-->978-0-14-012495-8}}</ref> The most famous and one of the most successful of these is [[tit-for-tat]] with a simple algorithm.

<syntaxhighlight lang="python">
def tit_for_tat(last_move_by_opponent):
    """Defect if opponent defects, else cooperate."""
    if last_move_by_opponent == defect:
        defect()
    else:
        cooperate()
</syntaxhighlight>

The pay-off for any single round of the game is defined by the pay-off matrix for a single round game (shown in bar chart 1 below). In multi-round games the different choices – co-operate or defect – can be made in any particular round, resulting in a certain round payoff.  It is, however, the possible accumulated pay-offs over the multiple rounds that count in shaping the overall pay-offs for differing multi-round strategies such as tit-for-tat.

[[File:PrisonersPayoff.jpg|thumb|400px|Payoffs in two varieties of prisoner's dilemma game
<br>Prisoner's dilemma: co-operate or defect
<br>Payoff <sub>(temptation in defecting vs. co-operation)</sub> > Payoff <sub>(mutual co-operation)</sub> > Payoff<sub>(joint defection)</sub> > Payoff<sub>(sucker co-operates but opponent defects)</sub>]]

Example 1: The straightforward single round prisoner's dilemma game.  The classic prisoner's dilemma game payoffs gives a player a maximum payoff if they defect and their partner co-operates (this choice is known as ''temptation''). If, however, the player co-operates and their partner defects, they get the worst possible result (the suckers payoff). In these payoff conditions the best choice (a [[Nash equilibrium]]) is to defect.

Example 2: Prisoner's dilemma played repeatedly. The strategy employed is ''tit-for-tat'' which alters behaviours based on the action taken by a partner in the previous round – i.e. reward co-operation and punish defection.  The effect of this strategy in accumulated payoff over many rounds is to produce a higher payoff for both players' co-operation and a lower payoff for defection.  This removes the temptation to defect.  The suckers payoff also becomes less, although "invasion" by a pure defection strategy is not entirely eliminated.

===Routes to altruism===

Altruism takes place when one individual, at a cost (C) to itself, exercises a strategy that provides a benefit (B) to another individual. The cost may consist of a loss of capability or resource which helps in the battle for survival and reproduction, or an added risk to its own survival. Altruism strategies can arise through:

{| class="wikitable"
|-
! Type !! Applies to: !! Situation !! Mathematical effect
|-valign="top"
| '''Kin selection''' – (inclusive fitness of related contestants) || Kin – genetically related individuals || Evolutionary game participants are genes of strategy. The best payoff for an individual is not necessarily the best payoff for the gene.  In any generation the player gene is ''not'' only in one individual, it is in a kin-group.  The highest fitness payoff for the kin group is selected by natural selection.  Therefore, strategies that include self-sacrifice on the part of individuals are often game winners – the evolutionarily stable strategy. Animals must live in kin-groups during part of the game for the opportunity for this altruistic sacrifice ever to take place.  || Games must take into account inclusive fitness.  Fitness function is the combined fitness of a group of related contestants – each weighted by the degree of relatedness – relative to the total genetic population.  The mathematical analysis of this gene-centric view of the game leads to Hamilton's rule, that the relatedness of the altruistic donor must exceed the cost-benefit ratio of the altruistic act itself:<ref name="Nowak & Sigmund">{{cite journal | last1=Nowak | first1=Martin A. | last2=Sigmund | first2=Karl | year=2005 | title=Evolution of indirect reciprocity | journal=Nature | volume=437 | issue=7063| pages=1293–1295 | doi=10.1038/nature04131 | pmid=16251955 |bibcode=2005Natur.437.1291N | s2cid=3153895 | url=http://pure.iiasa.ac.at/7763/1/IR-05-079.pdf }}</ref>

:''R>c/b''  R is relatedness, c the cost, b the benefit
|- valign="top"
| '''Direct reciprocity''' || Contestants that trade favours in paired relationships || A game theoretic embodiment of "I'll scratch your back if you scratch mine".  A pair of individuals exchange favours in a multi-round game. The individuals are recognisable to one another as partnered.  The term "direct" applies because the return favour is specifically given back to the pair partner only. || The characteristics of the multi-round game produce a danger of defection and the potentially lesser payoffs of cooperation in each round, but any such defection can lead to punishment in a following round – establishing the game as a repeated prisoner's dilemma.  Therefore, the family of tit-for-tat strategies come to the fore.<ref>{{cite book |author=Axelrod, R. |date=1984 |title=The Evolution of Cooperation |pages=Chapters 1 to 4 |publisher=Penguin |isbn=978-0-14-012495-8}}</ref>

|- valign="top"
| '''Indirect reciprocity''' || Related or non related contestants trade favours but without partnering.  A return favour is "implied" but with no specific identified source who is to give it. ||  The return favour is not derived from any particular established partner. The potential for indirect reciprocity exists for a specific organism if it lives in a cluster of individuals who can interact over an extended period of time.
It has been argued that human behaviours in establishing moral systems as well as the expending of significant energies in human society for tracking individual reputations is a direct effect of societies' reliance on strategies of indirect reciprocation.<ref>{{cite book |author=Alexander R. |title=The Biology of Moral Systems |date=1987 |publisher=Aldine Transaction |isbn=978-0-202-01174-5 |url-access=registration |url=https://archive.org/details/biologyofmoralsy0000alex }}</ref>
  
 || The game is highly susceptible to defection, as direct retaliation is impossible.  Therefore, indirect reciprocity will not work without keeping a social score, a measure of past co-operative behaviour. The mathematics lead to a modified version of Hamilton's rule where:
:''q>c/b'' where q (the probability of knowing the social score) must be greater than the cost benefit ratio<ref>{{cite journal | last1=Nowak | first1=Martin A. | s2cid=4395576 | year=1998 | title=Evolution of indirect reciprocity by image scoring | journal=Nature | volume=393 | issue=6685| pages=573–575 | doi=10.1038/31225 | pmid=9634232|bibcode=1998Natur.393..573N }}</ref><ref>{{cite journal | last1=Nowak | first1=Martin A. | last2=Sigmund | first2=Karl | year=1998 | title=The Dynamics of Indirect Reciprocity | journal=Journal of Theoretical Biology | volume=194 | issue=4| pages=561–574 | doi=10.1006/jtbi.1998.0775| pmid=9790830 | bibcode=1998JThBi.194..561N | citeseerx=10.1.1.134.2590 }}</ref>

Organisms that use social score are termed Discriminators, and require a higher level of cognition than strategies of simple direct reciprocity. As evolutionary biologist David Haig put it – "For direct reciprocity you need a face; for indirect reciprocity you need a name".
|}

===The evolutionarily stable strategy===
[[File:AssessorGraph.jpg|thumb|300px| The payoff matrix for the hawk dove game, with the addition of the assessor strategy. This "studies its opponent", behaving as a hawk when matched with an opponent it judges "weaker", like a dove when the opponent seems bigger and stronger. Assessor is an ESS, since it can invade both hawk and dove populations, and can withstand invasion by either hawk or dove mutants.]]
{{main|Evolutionarily Stable Strategy}}

The [[evolutionarily stable strategy]] (ESS) is akin to the Nash equilibrium in classical game theory, but with mathematically extended criteria. Nash equilibrium is a game equilibrium where it is not rational for any player to deviate from their present strategy, provided that the others adhere to their strategies. An ESS is a state of game dynamics where, in a very large population of competitors, another mutant strategy cannot successfully enter the population to disturb the existing dynamic (which itself depends on the population mix). Therefore, a successful strategy (with an ESS) must be both effective against competitors when it is rare – to enter the previous competing population, and successful when later in high proportion in the population – to defend itself. This in turn means that the strategy must be successful when it contends with others exactly like itself.<ref>Taylor, P. D. (1979). ''Evolutionarily Stable Strategies with Two Types of Players'' J. Appl. Prob. 16, 76–83.</ref><ref>Taylor, P. D., and Jonker, L. B. (1978). ''Evolutionarily Stable Strategies and Game Dynamics'' Math. Biosci. 40, 145–156.</ref><ref>Osborn, Martin, Introduction to Game Theory, 2004, Oxford Press, p. 393-403 {{ISBN|0-19-512895-8}}</ref>

An ESS is not:

* An optimal strategy: that would maximize fitness, and many ESS states are far below the maximum fitness achievable in a fitness landscape. (See hawk dove graph above as an example of this.)
* A singular solution: often several ESS conditions can exist in a competitive situation. A particular contest might stabilize into any one of these possibilities, but later a major perturbation in conditions can move the solution into one of the alternative ESS states.
* Always present: it is possible for there to be no ESS. An evolutionary game with no ESS is "rock-scissors-paper", as found in species such as the side-blotched lizard (''[[Uta stansburiana]]'').
* An unbeatable strategy: the ESS is only an uninvadeable strategy.

[[File:Agelenopsis actuosa fem sp.jpg|thumb|Female funnel web spiders (Agelenopsis aperta) contest with one another for the possession of their desert spider webs using the assessor strategy.<ref>{{cite journal | last1=Riechert | first1=S.|author1-link= Susan Riechert |last2= Hammerstein | first2=P. |year=1995 |title= Putting Game Theory to the Test |doi= 10.1126/science.7886443 | pmid=7886443 |journal= Science |volume= 267 | issue=5204 |pages= 1591–1593 |bibcode= 1995Sci...267.1591P | s2cid=5133742 }}</ref>]]

The ESS state can be solved for by exploring either the dynamics of population change to determine an ESS, or by solving equations for the stable stationary point conditions which define an ESS.<ref>{{cite journal | last1=Chen | first1=Z | last2=Tan | first2=JY | last3=Wen | first3=Y | last4=Niu | first4=S | last5=Wong | first5=S-M | year=2012 | title=A Game-Theoretic Model of Interactions between Hibiscus Latent Singapore Virus and Tobacco Mosaic Virus | journal=PLOS ONE | volume=7 | issue=5| page=e37007 | doi=10.1371/journal.pone.0037007 |bibcode=2012PLoSO...737007C | pmid=22623970 | pmc=3356392| doi-access=free }}</ref>  For example, in the hawk dove game we can look for whether there is a static population mix condition where the fitness of doves will be exactly the same as fitness of hawks (therefore both having equivalent growth rates – a static point). 
  
Let the chance of meeting a hawk=p so therefore the chance of meeting a dove is (1-p)

Let Whawk equal the payoff for hawk...

Whawk=payoff in the chance of meeting a dove + payoff in the chance of meeting a hawk

Taking the payoff matrix results and plugging them into the above equation:

{{math|<var>Whawk</var>{{=}} <var>V·(1-p)+(V/2-C/2)·p</var>}}

Similarly for a dove:

{{math|<var>Wdove</var>{{=}} <var>V/2·(1-p)+0·(p)</var>}}

so....

{{math|<var>Wdove</var>{{=}} <var> V/2·(1-p) </var>}}

Equating the two fitnesses, hawk and dove

{{math|<var>V·(1-p)+(V/2-C/2)·p</var>{{=}} <var> V/2·(1-p) </var>}}

... and solving for p

{{math|<var>p</var>{{=}} <var>V/C</var>}}

so for this "static point" where the ''population percent'' is an ESS solves to be ESS<sub>(percent Hawk)</sub>=''V/C''

Similarly, using inequalities, it can be shown that an additional hawk or dove mutant entering this ESS state eventually results in less fitness for their kind – both a true Nash and an ESS equilibrium.  This example shows that when the risks of contest injury or death (the cost C) is significantly greater than the potential reward (the benefit value V), the stable population will be mixed between aggressors and doves, and the proportion of doves will exceed that of the aggressors.  This explains behaviours observed in nature.

==Unstable games, cyclic patterns==

===Rock paper scissors===
[[File:Rock paper scissors.jpg|thumb|Rock paper scissors]]
[[File:RockPaperMatrix.jpg|thumb|Mutant invasion for rock paper scissors payoff matrix – an endless cycle]]
{{main|Rock paper scissors}}

[[File:RPS dynamics.jpg|thumb|A computer simulation of the rock scissors paper game. The associated [[Normal-form game|RPS game payoff matrix]] is shown. Starting with an arbitrary population the percentage of the three morphs builds up into a continuously cycling pattern.]]

Rock paper scissors incorporated into an evolutionary game has been used for modelling natural processes in the study of [[ecology]].<ref>Allesina and Levine, "A competitive network theory of species diversity", Proceedings of the National Academy of Sciences, 2011</ref> 
Using [[experimental economics]] methods, scientists have used RPS games to test human social evolutionary dynamical behaviours in laboratories. The social cyclic behaviours, predicted by evolutionary game theory, have been observed in various laboratory experiments.<ref>{{cite journal | last1=Hoffman | first1=M | last2=Suetens | first2=S | last3=Gneezy | first3=U | last4=Nowak | first4=M | year=2015 | title=An experimental investigation of evolutionary dynamics in the Rock-Paper-Scissors game | journal=Scientific Reports | volume=5 |  page=8817 | doi=10.1038/srep08817  |bibcode=2015NatSR...5E8817H | pmid=25743257 | pmc=4351537}}</ref><ref>{{cite journal | last1=Cason | first1=T | last2=Friedman | first2=D | last3=Hopkins | first3=E | year=2014 | title=Cycles and Instability in a Rock–Paper–Scissors Population Game: A Continuous Time Experiment | journal=Review of Economic Studies | volume=81 | issue=1 | pages=112–136 | doi=10.1093/restud/rdt023 | citeseerx=10.1.1.261.650 }}</ref>

===Side-blotched lizard plays the RPS, and other cyclical games===
<!-- [[Mimicry]] links to this section heading -->
The first example of RPS in nature was seen in the behaviours and throat colours of a small lizard of western North America. The [[side-blotched lizard]] (''Uta stansburiana'') is [[Polymorphism (biology)|polymorphic]] with three throat-colour morphs<ref>Sinervo, B. and Lively, C. M. 1996. The rock-scissors-paper game and the evolution of alternative male strategies. Nature 340:246</ref> that each pursue a different mating strategy:

[[File:SideblotchedLizard.jpg|thumb|left|The [[side-blotched lizard]] effectively uses a rock-paper-scissors mating strategy]]

* The orange throat is very aggressive and operates over a large territory – attempting to mate with numerous females 
* The unaggressive yellow throat mimics the markings and behavior of female lizards, and "sneakily" slips into the orange throat's territory to mate with the females there (thereby taking over the population)
* The blue throat mates with, and carefully guards, one female – making it impossible for the sneakers to succeed and therefore overtakes their place in a population
However the blue throats cannot overcome the more aggressive orange throats. Later work showed that the blue males are altruistic to other blue males, with three key traits: they signal with blue color, they recognize and settle next to other (unrelated) blue males, and they will even defend their partner against orange, to the death. This is the hallmark of another game of cooperation that involves a [[green-beard effect]].<ref>Sinervo, B. and Clobert, J. 2003. Morphs, dispersal, genetic similarity and the evolution of cooperation. [[Science (journal)|Science]] 300: 1949–1951</ref><ref>Sinervo, B., Chaine, A., Clobert, J., Calsbeek, R., McAdam, A., Hazard, H., Lancaster, L., Alonzo, S., Corrigan, G., and M. Hochberg. 2006a. Self-recognition, color signals and cycles of greenbeard mutualism and transient altruism. Proceedings of the National Academy of Sciences (U.S.A.). 102: 7372–7377</ref>

The females in the same population have the same throat colours, and this affects how many offspring they produce and the size of the progeny, which generates cycles in density, yet another game – the ''r-K'' game.<ref>Sinervo, B., Svensson, E. and Comendant, T. 2000. Density cycles and an offspring quantity and quality game driven by natural selection. Nature 406: 985–988</ref> Here, ''r'' is the [[Malthusian growth model|Malthusian parameter]] governing exponential growth, and ''K'' is the [[Carrying capacity|carrying capacity of the environment]]. Orange females have larger [[Clutch (eggs)|clutches]] and smaller offspring which do well at low density. Yellow & blue females have smaller clutches and larger offspring which do well at high density. This generates perpetual cycles tightly tied to population density. The idea of cycles due to density regulation of two strategies originated with rodent researcher [[Dennis Chitty]], ergo these kinds of games lead to "Chitty cycles". There are games within games within games embedded in natural populations. These drive RPS cycles in the males with a periodicity of four years and ''r-K'' cycles in females with a two year period.

The overall situation corresponds to the rock, scissors, paper game, creating a four-year population cycle. The RPS game in male side-blotched lizards does not have an ESS, but it has a Nash equilibrium (NE) with endless orbits around the [[Fixed point (mathematics)#Attracting fixed points|NE attractor]]. Following this Side-blotched lizard research, many other three-strategy polymorphisms have been discovered in lizards and some of these have RPS dynamics merging the male game and density regulation game in a single sex (males).<ref>Sinervo, B., Heulin, B., Surget-Groba, Y., Clobert, J., Corl, A., Chaine, A, and Davis, A. 2007. Models of density-dependent genic selection and a new Rock-Paper-Scissors social system. The American Naturalist, 170: 663–680.</ref> More recently, mammals have been shown to harbour the same RPS game in males and ''r-K'' game in females, with coat-colour polymorphisms and behaviours that drive cycles.<ref>Sinervo, B., A. Chaine, and D. B. Miles. Social Games and Genic Selection Drives Mammalian Mating System Evolution and Speciation. The American Naturalist 195:247–274.</ref> This game is also linked to the evolution of male care in rodents, and monogamy, and drives [[Speciation#Rates|speciation rates]]. There are ''r-K'' strategy games linked to rodent population cycles (and lizard cycles).<ref>Chitty, D. (1996). Do lemmings commit suicide?: beautiful hypotheses and ugly facts. Oxford University Press.</ref>

When he read that these lizards were essentially engaged in a game with a rock-paper-scissors structure, John Maynard Smith is said to have exclaimed "They have read my book!".<ref>Sigmund, Karl, " Interim Report IR-05-076 John Maynard Smith and Evolutionary Game Theory",  International Institute for Applied Systems Analysis, Dec. 2005</ref>

==Signalling, sexual selection and the handicap principle==
[[File:Peacock at Warwick Castle.jpg|thumb|The peacock's tail may be an instance of the [[handicap principle]] in action]]
{{main|Signalling theory}}

Aside from the difficulty of explaining how altruism exists in many evolved organisms, Darwin was also bothered by a second conundrum – why  a significant number of species have phenotypical attributes that are patently disadvantageous to them with respect to their survival – and should by the process of natural section be selected against – e.g. the massive inconvenient feather structure found in a peacock's tail.  Regarding this issue Darwin wrote to a colleague "The sight of a feather in a peacock's tail, whenever I gaze at it, makes me sick."<ref>Pallen, Mark, ''The Rough Guide to Evolution'', Penguin, 2009, p.74, {{ISBN|978-1-85828-946-5}}</ref>  It is the mathematics of evolutionary game theory, which has not only explained the existence of altruism, but also explains the totally counterintuitive existence of the peacock's tail and other such biological encumbrances.

On analysis, problems of biological life are not at all unlike the problems that define economics – eating (akin to resource acquisition and management), survival (competitive strategy) and reproduction (investment, risk and return). Game theory was originally conceived as a mathematical analysis of economic processes and indeed this is why it has proven so useful in explaining so many biological behaviours.  One important further refinement of the evolutionary game theory model that has economic overtones rests on the analysis of costs.  A simple model of cost assumes that all competitors suffer the same penalty imposed by the game costs, but this is not the case. More successful players will be endowed with or will have accumulated a higher "wealth reserve" or "affordability" than less-successful players.  This wealth effect in evolutionary game theory is represented mathematically by "[[resource holding potential]] (RHP)" and shows that the effective cost to a competitor with a higher RHP are not as great as for a competitor with a lower RHP.   As a higher RHP individual is a more desirable mate in producing potentially successful offspring, it is only logical that with sexual selection RHP should have evolved to be signalled in some way by the competing rivals, and for this to work this signalling must be done ''honestly''. [[Amotz Zahavi]] has developed this thinking in what is known as the "[[handicap principle]]",<ref>{{cite journal | last1=Zahavi | first1=A. | year=1975 | title=Mate selection – a selection for a handicap | journal=Journal of Theoretical Biology | volume=53 | issue=1| pages=205–214 | doi=10.1016/0022-5193(75)90111-3 | pmid=1195756| bibcode=1975JThBi..53..205Z | citeseerx=10.1.1.586.3819 }}</ref> where superior competitors signal their superiority by a costly display.  As higher RHP individuals can properly afford such a costly display this signalling is inherently honest, and can be taken as such by the signal receiver.  In nature this is illustrated than in the costly plumage of the [[peacock]].  The mathematical proof of the handicap principle was developed by [[Alan Grafen]] using evolutionary game-theoretic modelling.<ref name="Grafen 1990 517–546">{{cite journal |author-link=Alan Grafen |last=Grafen |first=A. |year=1990 |title=Biological signals as handicaps |journal=Journal of Theoretical Biology |volume=144 |issue=4 |pages=517–546 |doi=10.1016/S0022-5193(05)80088-8 |pmid=2402153|bibcode=1990JThBi.144..517G }}</ref>

==Coevolution==
{{main|Coevolution}}

Two types of dynamics:
* Evolutionary games which lead to a stable situation or point of stasis for contending strategies which result in an evolutionarily stable strategy
* Evolutionary games which exhibit a cyclic behaviour (as with RPS game) where the proportions of contending strategies continuously cycle over time within the overall population

{{multiple image
 | width=180
 | image1 =NHM_Taricha_granulosa_cropped.jpg
 | alt1=
 | caption1= Competitive Coevolution 
– The [[rough-skinned newt]] (''Tarricha granulosa'') is highly toxic, due to an [[evolutionary arms race]] with a predator, the [[common garter snake]] (''Thamnophis sirtalis''), which in turn is highly tolerant of the poison. The two are locked in a [[Red Queen's Hypothesis|Red Queen]] arms race.<ref>Pallen, M., ''Rough Guide to Evolution'', Penguin Books, 2009, p.123, {{ISBN|978-1-85828-946-5}}</ref> 
 | image2=NHM Xanthopan morgani.jpg
 | alt2=
 | caption2=Mutualistic Coevolution
– [[Darwin's orchid]] (''Angraecum sesquipedale'') and the moth [[Morgan's sphinx]] (''Xanthopan morgani'') have a mutual relationship where the moth gains pollen and the flower is [[pollination|pollinated]].
 }}

A third, [[coevolution]]ary, dynamic, combines intra-specific and inter-specific competition. Examples include predator-prey competition and host-parasite co-evolution, as well as mutualism.  Evolutionary game models have been created for pairwise and multi-species coevolutionary systems.<ref>Matja, Szolnoki, "Coevolutionary games – a mini review", Biosystems, 2009</ref> The general dynamic differs between competitive systems and mutualistic systems.

In competitive (non-mutualistic) inter-species coevolutionary system the species are involved in an arms race – where adaptations that are better at competing against the other species tend to be preserved. Both game payoffs and replicator dynamics reflect this. This leads to a [[Red Queen's Hypothesis|Red Queen]] dynamic where the protagonists must "run as fast as they can to just stay in one place".<ref>Cliff and Miller, "Tracking the red queen: Measurements of adaptive progress in co-evolutionary simulations", European Conference on Artificial Life, p. 200–218, 1995</ref>

A number of evolutionary game theory models have been produced to encompass coevolutionary situations. A key factor applicable in these coevolutionary systems is the continuous adaptation of strategy in such arms races. Coevolutionary modelling therefore often includes [[genetic algorithm]]s to reflect mutational effects, while computers simulate the dynamics of the overall coevolutionary game.   The resulting dynamics are studied as various parameters are modified.  Because several variables are simultaneously at play, solutions become the province of multi-variable optimisation. The mathematical criteria of determining stable points are [[Pareto efficiency]] and Pareto dominance, a measure of solution optimality peaks in multivariable systems.<ref>Sevan, Ficici and Pollack, "Pareto optimality in coevolutionary learning", European Conference on Artificial Life, pp. 316–325, 2001</ref>

[[Carl Bergstrom]] and Michael Lachmann apply evolutionary game theory to the division of benefits in [[Mutualism (biology)|mutualistic]] interactions between organisms. Darwinian assumptions about fitness are modeled using replicator dynamics to show that the organism evolving at a slower rate in a mutualistic relationship gains a disproportionately high share of the benefits or payoffs.<ref>{{cite journal | last1=Bergstrom | first1=C. | last2=Lachmann | first2=M. | year=2003 | title=The red king effect: when the slowest runner wins the coevolutionary race | journal= Proceedings of the National Academy of Sciences| volume=100 | issue=2| pages=593–598 | doi=10.1073/pnas.0134966100 | pmid=12525707 |bibcode=2003PNAS..100..593B | pmc=141041 | doi-access=free }}</ref>

==Extending the model==

A [[mathematical model]] analysing the behaviour of a system needs initially to be as simple as possible to aid in developing a base understanding the fundamentals, or “first order effects”, pertaining to what is being studied.  With this understanding in place it is then appropriate to see if other, more subtle, parameters (second order effects) further impact the primary behaviours or shape additional behaviours in the system.   Following Maynard Smith's seminal work in evolutionary game theory, the subject has had a number of very significant extensions which have shed more light on understanding evolutionary dynamics, particularly in the area of altruistic behaviors.  Some of these key extensions to evolutionary game theory are:

[[File:SpatialExamp.gif|thumb|upright|150px|''' A Spatial Game'''<br />In a spatial evolutionary game contestants meet in contests at fixed grid positions and only interact with immediate neighbors. Shown here are the dynamics of a Hawk Dove contest, showing Hawk and Dove contestants as well as the changes of strategy taking place in the various cells]]

=== Spatial games ===

Geographic factors in evolution include [[gene flow]] and [[horizontal gene transfer]].  Spatial game models represent geometry by putting contestants in a lattice of cells: contests take place only with immediate neighbours. Winning strategies take over these immediate neighbourhoods and then interact with adjacent neighbourhoods. This model is useful in showing how pockets of co-operators can invade and introduce altruism in the Prisoners Dilemma game,<ref>{{cite book |author=Nowak, Martin |title=Evolutionary Dynamics |url=https://archive.org/details/evolutionarydyna00nowa_778 |url-access=limited |date=2006 |publisher=Harvard University Press |pages=[https://archive.org/details/evolutionarydyna00nowa_778/page/n165 152]–154 |isbn=978-0-674-02338-3}}</ref> where Tit for Tat (TFT) is a Nash Equilibrium but NOT also an ESS. Spatial structure is sometimes abstracted into a general network of interactions.<ref>{{Cite journal | url=http://egtheory.wordpress.com/2012/03/21/spatial-structure/ | title=Statistical mechanics of complex networks| journal=Reviews of Modern Physics| volume=74| issue=1| pages=47–97| year=2002| last1=Albert| first1=Reka| last2=Barabasi| first2=Albert-Laszlo| doi=10.1103/RevModPhys.74.47| citeseerx=10.1.1.242.4753| bibcode=2002RvMP...74...47A| arxiv=cond-mat/0106096| s2cid=60545}}</ref><ref>H. Tembine, E. Altman, R. El Azouzi, Y. Hayel: Evolutionary Games in Wireless Networks. IEEE Transactions on Systems, Man, and Cybernetics, Part B 40(3): 634–646 (2010)</ref> This is the foundation of [[evolutionary graph theory]].

===Effects of having information===

In evolutionary game theory as in conventional [[game theory]] the effect of Signalling (the acquisition of information) is of critical importance, as in Indirect Reciprocity in Prisoners Dilemma (where contests between the SAME paired individuals are NOT repetitive). This models the reality of most normal social interactions which are non-kin related. Unless a probability measure of reputation is available in Prisoners Dilemma only direct reciprocity can be achieved.<ref name="Nowak & Sigmund" />  With this information indirect reciprocity is also supported.

Alternatively, agents might have access to an arbitrary signal initially uncorrelated to strategy but becomes correlated due to evolutionary dynamics. This is the [[green-beard effect]] (see side-blotched lizards, above) or evolution of ethnocentrism in humans.<ref>{{Cite journal | url=http://egtheory.wordpress.com/2012/06/13/evolution-of-ethnocentrism/ | doi=10.1177/0022002706293470| title=The Evolution of Ethnocentrism| journal=Journal of Conflict Resolution| volume=50| issue=6| pages=926–936| year=2006| last1=Hammond| first1=Ross A.| last2=Axelrod| first2=Robert| s2cid=9613947}}</ref> Depending on the game, it can allow the evolution of either cooperation or irrational hostility.<ref>Kaznatcheev, A. (2010, March). [http://www.cs.mcgill.ca/~akazna/kaznatcheev20100910.pdf Robustness of ethnocentrism to changes in inter-personal interactions]. In ''Complex Adaptive Systems–AAAI Fall Symposium''.</ref>

From molecular to multicellular level, a [[signaling game]] model with information asymmetry between sender and receiver might be appropriate, such as in mate attraction<ref name="Grafen 1990 517–546"/> or evolution of translation machinery from RNA strings.<ref>{{cite journal |author1=Jee, J. |author2=Sundstrom, A. |author3=Massey, S.E. |author4=Mishra, B.  | year=2013 | title= What can information-asymmetric games tell us about the context of Crick's 'Frozen Accident'? | journal=Journal of the Royal Society Interface | volume=10 | issue=88 | page=20130614 | doi=10.1098/rsif.2013.0614 | pmid=23985735 | pmc=3785830}}</ref>

===Finite populations===

Many evolutionary games have been modelled in finite populations to see the effect this may have, for example in the success of mixed strategies.

==See also==
{{Div col|colwidth=20em}}
*[[Adaptive dynamics]]
*[[Behavioral ecology]]
*[[Dynamical systems]]
*[[Evolutionary dynamics]]
*[[Gene-centered view of evolution]]
*[[Memetics]]
{{Div col end}}

==Notes==
{{Notelist}}

==References==
{{Reflist|30em}}

==Further reading==

* Davis, Morton; "Game Theory – A Nontechnical Introduction",  Dover Books, {{ISBN|0-486-29672-5}}
* {{cite book
 |last        = Dawkins
 |first       = Richard
 |author-link = Richard Dawkins
 |title       = [[The Selfish Gene]]
 |year        = 2006
 |edition     = 30th anniversary
 |location    = Oxford
 |publisher   = Oxford University Press
 |isbn        = 978-0-19-929115-1
}}
*Dugatkin and Reeve; "Game Theory and Animal Behavior", Oxford University Press, {{ISBN|0-19-513790-6}}
*Hofbauer and Sigmund; "Evolutionary Games and Population Dynamics", Cambridge University Press, {{ISBN|0-521-62570-X}}
* Kohn, Marek; "A Reason for Everything", Faber and Faber, {{ISBN|0-571-22393-1}}
* Li Richter and Lehtonen (Eds.) "Half a century of evolutionary games: a synthesis of theory, application and future directions", Philosophical Transactions of the Royal Society B, [https://royalsocietypublishing.org/toc/rstb/2023/378/1876 Volume 378, Issue 1876]
*Sandholm, William H.; "Population Games and Evolutionary Dynamics", The MIT Press, {{ISBN|0262195879|}}
*Segerstrale, Ullica; "Nature's Oracle – The life and work of W.D. Hamilton", Oxford University Press, 2013, {{ISBN|978-0-19-860727-4}}
*[[Karl Sigmund|Sigmund, Karl]]; "Games of Life",  Penguin Books, also Oxford University Press, 1993, {{ISBN|0198547838}}
* Vincent and Brown; "Evolutionary Game Theory, Natural Selection and Darwinian Dynamics", Cambridge University Press, {{ISBN|0-521-84170-4}}

==External links==
{{wikiquote}}
* [https://royalsocietypublishing.org/toc/rstb/2023/378/1876 Theme issue 'Half a century of evolutionary games: a synthesis of theory, application and future directions' (2023)]
* [http://plato.stanford.edu/entries/game-evolutionary/ Evolutionary game theory at the Stanford Encyclopedia of Philosophy]
* [https://web.archive.org/web/20070609172509/http://www.ethics.ubc.ca/eame/ Evolving Artificial Moral Ecologies at The Centre for Applied Ethics, University of British Columbia]
* {{cite web|url=https://www.youtube.com/playlist?list=PLVV0r6CmEsFzJSvAc4MBuUP_GrjO1lLHp|title=Life and work of John Maynard Smith, interviewed by Richard Dawkins|year=1997|work=[[Web of Stories]]|via=YouTube}} (via [https://www.webofstories.com/play/john.maynard.smith/1 Web of Stories])

{{Population genetics}}
{{Game theory}}
{{Authority control}}

{{DEFAULTSORT:Evolutionary Game Theory}}
[[Category:Evolutionary game theory| ]]
[[Category:Evolutionary dynamics]]
[[Category:Mathematical and quantitative methods (economics)]]