【转】演化博弈理论（EGT）

转载博文：http://blog.sina.com.cn/s/blog_71e1f5590102z5s1.html

演化博弈论是将博弈论运用到生物学中群体生命的演化问题，或用演化理论来发展博弈论。

演化博弈论（EGT）是定义一个竞争、策略和分析的框架，以期能够将达尔文的进化论模型化。EGT始于1973年，John Maynard Smith和George R. Price试图用“策略”分析将演化竞争形式化，并建立数学标准来预测不同竞争策略所产生的结果

 

演化博弈论与经典博弈论的不同之处在于EGT更为关注策略变化的动力学特性，即不仅考虑各种竞争策略本身优劣的影响，而且还要考虑竞争策略在群体中出现的频率的效果。

EGT对于解释许多复杂和具有挑战性的生物学问题极具价值，特别是它可以帮助我们解释在一个达尔文式的过程中如何建立利他行为。

不论其初衷如何，EGT理论引起了越来越多的经济学家、社会学家、人类学家和哲学家的兴趣。 

 

问题的提出

"一盎司代数学比一吨口头争论更有价值"  ---- J.B.S. Haldane （1892年11月5日－1964年12月1日，出生于大不列颠， 是一名遗传学家和演化生物学家，是种群遗传学的奠基人之一）

EGT理论来自一个具体的生物学问题： 如何解释动物在冲突情景中的仪式化行为。为什么有些动物在争夺资源中表现的非常”绅士“或”淑女“？这就是最前沿的动物行为学家Niko Tinbergen（1907年4月15日－1988年12月21日，荷兰动物行為学家与鸟类学家。1973年他与卡尔·冯·弗利、康拉德·洛伦兹因為在动物个体和群体行為的构成和激发方面做出了重大的贡献，共同获得诺贝尔生理学或医学奖）和Konrad Lorenz（1903年11月7日－1989年2月27日）是一位著名奥地利动物学家、鸟类学家、动物心理学家，也是经典比较行为研究的代表人物，受他的老师奥斯卡·海因洛斯的影响，建立了现代动物行為学。1973年他和卡尔·冯·弗利、尼可拉斯·庭伯根共同获得诺贝尔生理学或医学奖）所考虑的问题。

Tinbergen的解释是：这类行为之所以存在是因为对整个物种有利。但Maynard Smith看不出这种说法如何能与达尔文的思想相容，因为按照达尔文的理论，自然选择发生在个体层面，所以自利行为会被奖励而利他（以及群体）的行为并不会得到奖励。Maynard Smith过去是名工程师和能力极强的数学家，转而运用博弈论来回答这个问题。可以用博弈论解决此问题的建议最初是George Price（October 6, 1922 – January 6, 1975)，美国人口遗传学家，最初是名物理化学家，后来成了一名科学记者。他1967年移居伦敦后在Galton Lab理论生物工作室工作，有三项重大贡献：1、种群选择理论的Price方程；2、演化稳定策略（ESS），这是博弈论的一个中心概念；3、将Fisher的自然选择基础定理的形式化）。Maynard Smith的数学解释要现实的多，成了理论生物学中的一项重大数学贡献。

​

经典博弈论

博弈论由John von Neumann提出，为的是解释竞争博弈中的双方如何决定最佳策略。一个博弈会涉及若干玩家，其中每一方都面临策略选择。博弈游戏可以是一局，也可以是多局重复的。玩家如何玩此游戏构成玩家的”策略“。游戏规则决定玩家每一种策略所得到的回报（好处）。这些规则和相应的回报可以由一个回报矩阵（Payoff Matrix）中的决策树来表达。

经典博弈论本质上要求每一个玩家进行理性选择（即根据对结果评估结果进行全理性策略选择）​，其结果是经典博弈论要求每一个玩家都必须对对手的策略进行分析并在此基础上决定自己的策略是否合理。

​

从经典博弈论到进化博弈论

经过深入思考，Maynard Smith意识到在进化博弈论中并不需要每一玩家都理性行事，他们是需要有个策略而已。游戏的结果将会检验这个策略是好是坏，这就是演化的含义：通过演化可以检验不同策略在环境中的生存和复制能力。在生物学中，个体的博弈策略基本上是通过基因遗传决定的特性，而策略是由算法决定的，正如计算机程序那样。进化博弈论中的关键点是：一种策略的成功并不是仅仅由策略本身的好坏决定的，而是它在一组可选策略中有多好，以及其它策略在一个竞争性的群体中出现的频率。同时这也是一个策略与自己对玩能有多好，因为在生物界中，一个成功的策略将最终成为整个群体的主导策略，而群体中的个体将不得不面对他人采用的完全相同的策略。（思考：采用民主宪政策略的国家在起初面对非民主宪政国家时也许会获得竞争优势，但当各国均民主宪政时，这种策略还将是好的策略吗？）

进化博弈游戏中的参与者的目标是变得比其他竞争者更好，例如复制更多的下一代，其回报可以用适应度（Fitness，繁殖能力的相对值）来衡量。博弈游戏总是在一个极大的竞争群体中进行的多玩家游戏。除了在经典博弈论中的那些规则之外，进化博弈论的游戏规则还要包括基因复制动力学规则，即环境的生存适应者如何在群体中扩散自己的基因以及不适应环境这如何从群体中被剔除（用基因复制方程来表达）。基因复制动力学本质上就是对遗传机制进行建模，但为了简单起见，没有考虑变异问题。类似地，为了公式简单进化动力学只考虑无性繁殖情形。游戏可以重复迭代下去而不附加终止条件。分析游戏的结果包括：群体变化的动力学特性、成功的（活下来的）策略以及最终形成的各种平衡态。和经典博弈论不同的是，游戏的玩家并不选择他们的策略，也没有能力去改变其策略，他们的策略是与生俱来的而且他们的后代将继承相同的策略。

​

模型

进化博弈论模型

Evolutionary game theory transposes Darwinian mechanisms into a mathematical form by adopting a System Model of evolutionary processes with three main components - Population, Game, and Replicator Dynamics. The system process itself has four phases:

1) The model (as evolution itself) deals with a Population (Pn). The population will exhibit Variation among Competing individuals. In the model this competition is represented by the Game.

2) The Game tests the strategies of the individuals under the “rules of the game”. These rules produce different payoffs – in  units of Fitness (the production rate of offspring). The contesting individuals meet in pairwise contests with others, normally in a highly mixed distribution of the population. The mix of strategies in the population affects the payoff results by altering the odds that any individual may meet up in contests with various strategies. The individuals leave the game pairwise contest with a resulting fitness determined by the contest outcome – generally represented in a Payoff Matrix.

3) Based on this resulting fitness each member of the population then undergoes replication or culling determined by the exact mathematics of the Replicator Dynamics Process. This overall process then produces a New Generation P(n+1). Each surviving individual now has a new fitness level determined by the game result.

4) The new generation then takes the place of the previous one and the cycle begins again (and never stops). Mathematically speaking it is an Iterative process. Over time the population mix in such a system may converge to a stationary state – and if such a state cannot be invaded by any new “mutant strategies” it is by definition an Evolutionary Stable State (ESS)

It is important to realize that EGT is not just a specialist mathematical treatment of animal contests to determine their dynamics and results, but in a manner similar to the field of Evolutionary Algorithms, it is a comprehensive model that encompasses the Darwinian process itself – including the central tenets of competition (the game) natural selection (replicator dynamics) and heredity - all within the overall model. Therefore, it is a major vehicle to help understand and explain some of the most fundamental questions in biology including the issue of group selection, sexual selection, altruism, parental care, co-evolution, and ecological dynamics. Much of the progress in developing understanding in these diverse areas has been aided by evolutionary game theory modelling and many of the counter intuitive situations in these areas have been explained and put on a firm mathematical footing by the use of these models.

The common methodology to study the evolutionary dynamics in games is through replicator equations. These replicator equations in the context of evolutionary biology show the growth rate of the proportion of organisms using a certain strategy and that rate is equal to the difference between the average payoff of that strategy and the average payoff of the population as a whole. Continuous replicator equations assume infinite populations, continuous time, complete mixing and that strategies breed true. The attractors (stable fixed points) of the equations are equivalent with evolutionarily stable states. A strategy which can survive all "mutant" strategies is considered evolutionary stable. In the context of animal behavior, this usually means such strategies are programmed and heavily influenced by genetics, thus making any player or organism's strategy determined by these biological factors.

​

博弈

In Game Theory various games are mathematical objects of different character — in rules, relative payoffs, and attendant mathematical behaviours. Each "game" represents different situations — in the kinds of problems that organisms have to deal with, and the possible strategies that they might adopt if they are to successfully survive and reproduce. To achieve a better feel for the challenges of these different situations, evolutionary games are often given rather colourful names and “cover stories” which quite effectively describe the general situation in which the particular game places its players. It all helps develop a feel for the mathematics of the game and the problems the players face.

Some representative games of evolutionary game theory are hawk-dove, war of attrition, stag hunt, producer-scrounger, tragedy of the commons, and prisoner's dilemma. Some of the various strategies that apply in these games are Hawk, Dove, Bourgeois, Prober, Defector, Assessor, and Retaliator. Depending on the particular “Game” the various strategies vie against one another under the particular game rules, and the mathematics of the evolutionary game theory are used to determine the results and behaviours.

​

老鹰、鸽子博弈（懦夫博弈）

胆小鬼博弈（英文：The game of chicken），又译懦夫博弈，是博弈论中一个影响深远的模型。模型中，两名车手相对驱车而行，谁最先转弯的一方被耻笑為「胆小鬼」（chicken），让另一方胜出，因此这博弈模型在英文中称為The Game of Chicken(懦夫游戏），但如果两人拒绝转弯，任由两车相撞，最终谁都无法受益。这套模型在政治、经济上经常使用，也被用来形容相互保证毁灭的核战争，其中1962年古巴导弹危机常列入胆小鬼博弈的典型例子。

Solution for V=2, C=10 and fitness starting base B=4. The fitness of a Hawk for different population mixes is plotted as a black line, that of Dove in red. An ESS (a stationary point) will exist when Hawk and Dove fitness are equal: Hawks are 20% of population and Doves are 80% of the population.

The most classic game (and Maynard Smith's starting point) is the Hawk Dove game. The game was conceived to analyse the animal contest problem highlighted by Lorenz and Tinbergen. It is a contest over a shareable resource. The contestants can be either a Hawk or a Dove. These are not two separate species of bird; they are two subtypes of one species with two different types of strategy (two different morphs). The term Hawk Dove was coined by Maynard Smith because he did his work during the Vietnam War when political views fell into one of these two camps. The strategy of the Hawk (a fighter strategy) is to first display aggression, then escalate into a fight until he either wins or is injured. The strategy of the Dove (fight avoider) is to first display aggression but if faced with major escalation by an opponent to run for safety. If not faced with this level of escalation the Dove will attempt to share the resource.

Given that the resource is given the value V, the damage from losing a fight is given cost C:

• If a Hawk meets a Dove he gets the full resource V to himself

• If a Hawk meets a Hawk – half the time he wins, half the time he loses…so his average outcome is then V/2 minus C/2
• If a Dove meets a Hawk he will back off and get nothing - 0
• If a Dove meets a Dove both share the resource and get V/2

 

The actual payoff however depends on the probability of meeting a Hawk or Dove, which in turn is a representation of the percentage of Hawks and Doves in the population when a particular contest takes place. But that population makeup in turn is determined by the results of all of the previous contests before the present contest- it is a continuous iterative process where the resultant population of the previous contest becomes the input population to the next contest. If the cost of losing C is greater than the value of winning V (the normal situation in the natural world) the mathematics ends in an ESS – an evolutionarily stable strategy situation having a mix of the two strategies where the population of Hawks is V/C. The population will progress back to this equilibrium point if any new Hawks or Doves make a temporary perturbation in the population. The solution of the Hawk Dove Game explains why most animal contests involve only “ritual fighting behaviours” in contests rather than outright battles. The result does not at all depend on “good of the species” behaviours as suggested by Lorenz, but solely on the implication of actions of “selfish genes”.

​

消耗战博弈

In the Hawk Dove game the resource is sharable, which gives payoffs to both Doves meeting in a pairwise contest. In the case where the resource is not sharable but an alternative resource might be available by backing off and trying elsewhere, pure Hawk or Dove strategies become less effective. If an unshareable resource is combined with a high cost of losing a contest (injury or possible death) both Hawk and Dove payoffs are then further diminished. A safer strategy of lower cost display, bluffing and waiting to win, then becomes viable – a Bluffer strategy. The game then becomes one of accumulating costs, either the costs of displaying or costs of prolonged unresolved engagement. It’s effectively an auction; the winner is the contestant who will swallow the greater cost while the loser gets, for all his pains, the same cost as the winner but NO resource.

 The resulting evolutionary game theory mathematics leads to an optimal strategy of timed bluffing.

 

不同资源价值下的消耗战。Note the time it takes for an accumulation of 50% of the contestants to quit vs. the Value(V) of resource contested for

This is because in the war of attrition any strategy that is unwavering and predictable is unstable – unstable in the sense that such a strategy will ultimately be displaced by a mutant strategy which will simply rely on the fact that it can best the existing predictable strategy just by investing just an extra small delta of waiting resource to insure that it wins. Therefore, only a random unpredictable strategy can maintain itself in a population of Bluffers. The contestants in effect choose an “acceptable cost” to be incurred related to the value of the resource being sought – randomly selected at contest start - effectively making a related random“bid”which becomes part of a mixed strategy (a strategy where a contestant has several, or even many, possible actions in his strategy). This implements a distribution of bids for a resource of specific value V, where the particular bid made for any specific contest is chosen at random from within that distribution. The distribution (an ESS) can be computed by invoking the Bishop-Cannings theorem, which holds true for any case of mixed strategy ESS.

 The distribution function in these contests was thus determined by Parker and Thompson to be:

which is shown in the adjacent graph. The intuitive sense that greater values of resource sought leads to greater waiting times is borne out here.

粪蝇 (Scatophaga stercoraria) - a War of Attrition player

螳螂虾（富贵虾） guarding its home playing the Bourgeois Strategy

 

动物策略的例子:  by examining the behaviours, then determining both the Costs and the Value of resources attained in a contest the strategy of an organism can be verified.

This is exactly what is observed in nature for contests in a number of species, for example between male dung flies contesting for mating sites.

 The timing of disengagement in these contests follows the exact mathematical curve derived from the evolutionary theory mathematics.

 

允许新策略的不对称性

In the War of Attrition there must be nothing that signals the size of a bid to an opponent, otherwise this can act as a cue which can be utilised by an opponent for an effective counter-strategy for a mutant. There is however a mutant strategy which can better a Bluffer in the War of Attrition Game if such an asymmetry exists. This mutant strategy is the Bourgeois strategy (Maynard Smith named the strategy Bourgeois, because with his background of communism he regarded it as “politically bourgeois” way to value ownership). Bourgeois uses an asymmetry of some sort to break the deadlock. In nature one such asymmetry is possession - which contestant has the prior possession of the resource. The strategy is to play a Hawk if in possession of the resource but to display then retreat if not in possession. This strategy requires greater cognitive capability than Hawk but Bourgeois is still a very prevalent strategy in many animal contests… some examples being mantis shrimp and speckled wood butterfly contests. However, studies done on contests between speckled wood butterflies have been criticized for the methods used. Butterflies that were initially resource owners were held away from their territories very briefly in nets, a method that is suggested to tax the owners' metabolisms and potentially cause them to lose ensuing contests. Thus, evolutionary game theory explains a very wide range of rather mystifying behaviours in animals and in plant contests for many species with an illuminating clarity.

​

社会行为几种可选策略

 

社会互动博弈的几种策略  

 

Games like Hawk Dove and War of Attrition represent pure competition between individuals and have no attendant social element to the game. Where social influences apply there are four possible alternatives for strategic interaction that exist for the competitors. This is shown on the adjacent figure, where a plus sign represents a benefit and a minus sign represents a cost.

• 在合作或互利关系中，both "donor" and "recipient" are almost indistinguishable as both gain a benefit in the game by co-operating, i.e. the pair are in a game-wise situation where both can gain by executing a certain strategy, or alternatively both must act in concert because of some encompassing constraints that effectively puts them "in the same boat".
• ​

• 在利他关系中， the donor, at a cost to himself provides a benefit to the recipient. In the general case the recipient will have a kin relationship to the donor and the donation is one-way. Behaviours where benefits are donated alternatively (in both directions) at a cost, are often called altruistic, but on analysis such "altruism" can be seen to arise from optimised "selfish" strategies
• ​

• 损人损己（Spite） is essentially a“reversed”form of altruism where an ally is aided by damaging the ally's competitor(s). The general case is that the ally is kin related and the benefit is an easier competitive environment for the ally. Note: George Price, one of the early mathematical modellers of both altruism and spite, found this equivalence particularly disturbing at an emotional level.
• ​

• 自私性  is the base criteria of all strategic choice from a game theory perspective – strategies not aimed at self-survival and self-replication are not long for any game. Critically however, this situation is impacted by the fact that competition is taking place on multiple levels - i.e. at a genetic, an individual and a group level.

The rationale and the mathematics that lie behind adopting one of these alternative social strategies will be covered in the following sections of this article.

​

谁在博弈？

 

The Belding's ground squirrel lives in communities of closely related females and their young and male “immigrants”. This is so because males leave the colony on reaching maturity and find other colonies to join. When predators are in the vicinity of a colony certain squirrels emit a loud piercing alarm call, allowing other colony members to take cover. This call substantially endangers the caller as it easily locates it for the predator. However, as female squirrels are so closely related evolutionary game theory utilising measures of Inclusive Fitness shows that this behaviour is superior to not calling for them. Field studies confirm this is exactly how the females behave. The males, however, having no such level of inclusive fitness, in general do not call.

At first glance it may be surmised that the contestants of evolutionary games are the individuals present in each generation who directly participate in the game. On reflection, however, we see that individuals live only through one game cycle, and instead it is the strategies that really contest with one another over the duration of these recursive games. So it is ultimately genes that play out a full contest – genes of STRATEGY (i.e. selfish genes). The contesting genes are not just present in an individual and his/her direct linage; they are also present to a relational degree in all of the individual’s kin. This can sometime profoundly affect the kinds of strategies that will survive, and nowhere is this more pertinent than in issues of cooperation and defection. William Hamilton,[15] whose work defined kin selection, was involved in much of the groundbreaking mathematical work in this area and Hamilton treated many of these cases using game theoretic models. Kin related treatment of game contests[16] help to explain many aspects of the behaviours of eusocial insects, the altruistic behaviour in parent/offspring interactions, mutual protection behaviours, and co-operative care of offspring. For such games Hamilton defined an extended form of fitness that is operative – Inclusive fitness, which extends the fitness measure to include an individual’s offspring as well as any other “offspring equivalents” found in kin.

​

亲缘选择的数学表达

The concept of Kin Selection can be formulated into a mathematical relationship by expressing the basic definition that:

inclusive fitness = own contribution to fitness + contribution of all relatives

.

Fitness is measured relative to the average population at large; for example, fitness = 1 means growth at the same average rate of the population, fitness < 1 means decreasing percentage-wise in the population (dying out), fitness> 1 means increasing percentage-wise in the population (taking over).

The inclusive fitness of an individual wi is the sum of its specific fitness of itself ai plus the specific fitness of each and every relative weighted by the degree of relatedness which equates to the summation of all rj*bj....... where rj is relatedness of a specific relative and bj is that specific relative's fitness - producing:

Now if individual ai sacrifices his "own average equivalent fitness of 1" by accepting a fitness cost C, and then to "get that loss back", wi must still be 1 (or greater than 1)...AND if we use R*B to represent the summation we get:

1< (1-C) + RB  or rearranging... R>C/B

 

Hamilton then went on beyond areas of kin relatedness to work with Robert Axelrod  to analyze games that involved co-operation under conditions not involving kin where reciprocal altruism comes into play.[18]

食肉蚁 - Iridomyrmex purpureus-Relatedness of a worker ant (always a female) To mother or father=.5 To sister+=.75 To own daughter or son=.5 To brother=.25....... Therefore, it is more advantageous genetically to help produce a sister (.75) than to have a child oneself (.5). Furthermore, since ALL females in colony are sisters, this insures that new colonies that disperse from the existing one will also have strong kin related linkages - further multiplying the benefit to cost ratio of altruism

真社会性（Eusociality）及亲缘选择

 

“Ants are good citizens, they place group interests first” Clarence Day

Insect societies have always been a source of fascination and inscrutability. Certainly one of the very most inexplicable behaviours of these eusocial insects is the forfeiture of reproductive rights that the workers grant in favour of the queen. In a Darwinian sense, no greater sacrifice can ever exist. The explanation that has been forthcoming is in Kin Selection influences that arise from the genetic makeup of these workers which predisposes them to such kin oriented altruistic behaviours .[19] Most eusocial insect societies have haplo-diploid sexual determination, which in essence means that males develop from unfertilised eggs, females from fertilised. This leads to the situation in these Haplodiploid species, that sisters share 75% of their genes in common…. in effect more than they genetically share with their mother. But in this system ALL the other workers are sisters, further skewing the kin related payoffs toward altruistic sacrifice. Helping to raise the queen's offspring, which are also sisters, therefore has major payoffs as does sacrificing onself if the nest is invaded by some predator. For those insect societies which do not exhibit haplodiplodal genetic relationships, the most significant example being termites, there still exist kin related influences arising from inbreeding, monogamy and the ineffectiveness of dispersal.

The widely accepted kin related explanation of insect eusociality has however been challenged recently by a few highly noted evolutionary game theorists (Nowak and Wilson) who have published a controversial alternative game theoretic explanation based on a sequential development and group selection effects proposed for these insect species.

​

囚徒困境

One of the great difficulties of Darwinian Theory, and one recognised by Darwin himself was the problem of altruism. If the basis for selection is at the individual level, altruism makes no sense at all. But universal selection at the group level (for the good of the species, not the individual) fails to pass the test of the mathematics of game theory and is certainly not found to be the general case in nature.

 Yet in many social animals altruistic behaviour can be found. The solution to this paradox can be found in the application of evolutionary game theory to the “Prisoners Dilemma” game - a game which tests the payoffs of cooperating or in defecting from cooperation. It is certainly the most studied game in all of Game Theory.

As with all games in evolutionary game theory the analysis of Prisoners Dilemma is as a repetitive game. This repetitive nature affords competitors the possibility of retaliating for “bad behaviour” (defection) in previous rounds of the game. There is a multitude of strategies which have been tested by the mathematics of EGT and in computer simulations of contests and the conclusion is that the best competitive strategies are general cooperation with a reserved retaliatory response if necessary.

 The most famous and certainly one of the most successful of these strategies is Tit for Tat which carries out this approach by executing a simple algorithm.

The pay-off for any single round of the game is defined by the pay-off matrix for a single round game (shown in bar chart 1 below). In multi-round games the different choices - Co-operate or Defect - can be made in any particular round, resulting in a certain round payoff. It is, however, the possible accumulated pay-offs over the multiple rounds that count in shaping the overall pay-offs for differing multi-round strategies such as Tit-for-Tat.

两种囚徒困境博弈中各种决策的回报图

 囚徒困境：合作或出卖？ Two robbers that have promised never to “snitch” on each other are arrested, but police don’t have enough evidence for a maximum sentence. If the robbers both stick to their promise and don’t snitch on each other the police can only charge and convict them both with a lesser sentence. The robbers are separated and interrogated in different rooms where both are offered the chance at “turning state’s evidence” - escaping completely any sentence for himself, but sticking his partner with the maximal sentence . Each worries that even if he keeps his pledge, his partner will snitch and make him a total sucker for being so true to his word and landing himself a maximum sentence. 

The characteristic of the payoffs for Prisoners Dilemma in any individual round of play are: 

Payoff (Temptation in Defecting vs. Co-operation) >Payoff (Mutual Co-operation) > Payoff(Joint Defection) > Payoff (Sucker - you co-operate but opponent defects)​

例一:

 This is the straightforward single round Prisoners Dilemma Game (the game is executed in only one round). The classic Prisoners Dilemma game payoffs gives you a maximum payoff if you defect and your "partner" co-operates (this choice for you is known as TEMPTATION). If however you co-operate and your partner defects you get the worst possible result ( the SUCKERS PAYOFF ). In these payoff conditions the best choice (a Nash Equilibrium in classic game theory) is to defect.

例二:

 This is Prisoners Dilemma played repeatedly. The strategy employed is Tit-for-Tat which alters behaviors based on the action taken by a partner in the previous round - i.e. reward co-operation and punish defection. The effect of this strategy in ACCUMULATED payoff over many rounds is to produce a higher payoff for both players co-operation and a lower payoff for defection. This removes the Temptation to defect. The suckers payoff also becomes less, although "invasion" by a pure defection strategy is not entirely eliminated (this can be overcome by clustering of cooperative strategies - see spatial games below).

​

通往利他的路径

 

Altruism takes place when one individual, at a cost C to itself, exercises a strategy that provides a benefit B to another individual. The cost may consist of a loss of capability or resource which helps in the “battle for survival and reproduction” or it may consist of an added risk to its own survival – i.e. incurring a higher probability of non-survival for itself. Altruism strategies arise in a number of ways:

类型 1：亲缘选择– (相关竞争者的内含适应度)

应用于：亲缘 – 血缘相关的个体

情景：进化博弈的参与者实际上是“策略的基因。所以对个体回报最好的策略不一定是对基因回报最好的策略。在任何一代中，参与者的基因不仅在个体身上，它还存在于整个亲缘组中。对亲缘组织回报最多的策略就是较好的策略，就会被自然选择。因此，包括个体“自我牺牲”的策略往往就是游戏的赢家，这也是稳定策略 – 当然，动物在一生的某个阶段（或一生中）必须在血缘群体中生存，这样才可能有机会让利他牺牲发生。

数学结果 ：博弈必须考虑内含适应度。适应度方程描述的是整个群组的或相关参与者的适应度 – 每个个体都按相关性加权 – 即与整个基因群体的相关度。这种以基因为中心的博弈观点的数学分析可以推导出哈“密尔顿法则”，即利他提供者的基因相关度必须大于利他行为本身的成本-利益系数，数学表示为： R>c/b  （其中：R是相关度，c是成本，b是利益）

​

​类型 2：直接互惠

应用于：Related or non related contestants that “trade favours” in paired relationships

情景：A game theoretic embodiment of the adage “I’ll scratch your back if you scratch mine”. A pair of individuals exchange favours in a multi-round game in which both individuals are involved. The individuals in the pair are specifically recognisable to one another as partnered. The term “DIRECT” applies because the return favour is specifically given back to the pair partner only.

数学结果：The characteristics of the multi-round game produce a danger of “defection” and the potentially lesser payoffs of “cooperation” in each round, however any such defection can lead to subsequent "punishment" in a following round – establishing the game as that of classical repeated Prisoners Dilemma. Therefore, the family of tit-for-tat strategies come to the fore.[27]

 

类型 3：间接互惠

应用于：Related or non related contestants “trade favours” but not with partnering involved. A return favour is “implied” but with no specific identified source who is to give it.

情景：This behaviour is akin to “I’ll scratch your back, you scratch someone elses back, another someone else will scratch mine (probably)". The return favour is not derived from any particular established partner. The potential for indirect reciprocity exists for a specific organism if it lives in a cluster of individuals who can interact over an extended period of time.

It has been argued that human behaviours in establishing moral system as well as the expending of significant energies in human society for tracking individual reputation is a direct effect of societies reliance on strategies of indirect reciprocation.[28]

数学结果：The game is highly susceptible to defection, particularly as direct retaliation becomes essentially impossible. Therefore, indirect reciprocity will not work mathematically unless some additional factor is at play to verify that any individual being granted a favour is “trustworthy” to return the favour in the sharing group and insuring the punishment (non-reward) of defectors for “misbehaving”. This requires essentially keeping a "social score", a measure of the past co-operative behaviour. The mathematics leads to a modified version of Hamilton's Rule where: q>c/b where q (the probability of knowing the social score) must be greater than the cost benefit ratio

​

Organisms that use this sort of social score as an extended modification of the tit-for-tat variant strategies are termed Discriminators. Different discrimination algorithms exist for determining social scores, but in general all of these require a much higher level of cognition to exist than strategies of simple direct reciprocity. As David Haig, a noted evolutionary biologist succinctly put it - "For direct reciprocity you need a face; for indirect reciprocity you need a name".

 

ESS： 演化稳定策略

 (ESS) is perhaps the most widely known albeit most widely misunderstood concept in evolutionary game theory. The ESS is basically akin to Nash Equilibrium in classical Game Theory, but with mathematically extended criteria.

老鹰、鸽子游戏的回报矩阵附加评估者策略. The Assessor competitor "studies its opponent" and behaves as a Hawk when matched with an opponent it judges "weaker", and behaves like a Dove when it assesses its opponent as stronger. (The usual judgement criteria in nature is opponent size) Assessor is an ESS. As can be seen in the payoffs of the matrix it can invade as a mutant into both Hawk and Dove populations, and can ALSO withstand invasion by either Hawk or Dove mutants as it gets a better payoff when matched against another Assessor

Nash Equilibrium is a game equilibrium where it is not rational for any player to deviate from the present strategy they are executing. As discussed, in Evolutionary game Theory contestants are NOT behaving with rational choice, nor do they have the ability to totally alter their strategy, aside from executing a very limited “mixed strategy”. An ESS is instead a state of game dynamics where, in a very large (or infinite) population of competitors, another mutant strategy cannot successfully enter the population to disturb the existing dynamic (which in itself is population mix dependent). This leads to a situation where to be a successful strategy having an ESS, the strategy must be both effective against competitors when it is rare - to enter the previous competing population, and also successful when later in high proportion in the population - to “defend itself”.[31]

 This in turn necessarily means that the strategy needs to be successful when it contends with others exactly like itself. ESS is NOT:

• An OPTIMAL strategy – an optimal strategy would maximize Fitness, and many ESS states are far below the maximum fitness achievable in a fitness landscape. (see Hawk Dove graph above as an example of this)

•  
• A singular solution – often several ESS conditions can exist in a competitive situation. A particular contest might stabilize into any one of these possibilities, but later a major perturbation in conditions can move the solution into one of the alternative ESS states.
• ​
• Always present - It is also possible for there to be no ESS. An example evolutionary game with no ESS is the Rock-Scissors-Paper game found in a number of species (an example the side-blotched lizard (Uta stansburiana

))

• An unbeatable strategy - The ESS strategy is not necessarily an unbeatable strategy, it is only an uninvadable one.
• ​

评估者策略玩家- - Female funnel web spiders (Agelenopsis aperta) contest with one another for the possession of their desert spider webs. A higher-quality established web location offers a considerable reproductive advantage and is well worth holding, but serious fights for them are potentially very dangerous for the contestants. The spiders have been found to use the Assessor strategy, with size the critical determinant of winner. When an intruder spider enters a web to contest for possession the spiders become involved in a “web bouncing” behaviour which in essence establishes size by "weighing" the two contestants. The smaller spider, even if the original possessor of the web, leaves the site of the contest. In a detailed study Riechert proved that an Assessor "weighing" strategy was in play by adding a ballast weight to physically smaller spiders.[32] 

The ESS state can be solved for mathematically by exploring either the dynamics of population change to determine any ESS.... or alternatively by solving equations for the stable stationary point conditions which fundamentally define an ESS.[33] For example, in the Hawk Dove Game we can look for whether there is a static population mix condition where the fitness of Doves will be exactly the same as fitness of Hawks (therefore both having equivalent growth rates - a "static point").

Let chance of meeting a Hawk=p so therefore the chance of meeting a dove is (1-p)

Let WHawk equal the Payoff for Hawk.....

WHawk=Payoff in the chance of meeting a Dove + Payoff in the chance of meeting a Hawk

Taking the PAYOFF MATRIX results and plugging them into the above equation:

WHawk = V·(1-p)+(V/2-C/2)·p

Similarly for a Dove:

WDove = V/2·(1-p)+0·(p)

so....

WDove = V/2·(1-p)

Equating the two fitnesses, Hawk and Dove

V·(1-p)+(V/2-C/2)·p= V/2·(1-p)

... and solving for p

p = V/C

so for this "static point" where the Population Percent is an ESS solves to be ESS(percent Hawk) =V/C

Similarly using inequalities it can be shown that an additional Hawk or Dove “mutant” entering this ESS state generates a situation leading eventually to LESS fitness for their kind – both a true Nash and an ESS equilibrium. This fairly simple example shows that when the risks of contest injury or death (the Cost C) is significantly greater than the potential reward offered (the benefit value V) then the stable population which is reached will be MIXED between the aggressors and the doves, and that the proportion of doves will exceed that of the aggressors. This then mathematically explains behaviours that are actually observed in nature.

​

石头、纸头、剪刀游戏

                                                                                   石头、纸头、剪刀游戏

 

                                                                 石头、纸头、剪刀回报矩阵的突变体侵入 - 永恒的周期

 

An evolutionary game that actually turns out to be a children’s game is rock-paper-scissors. The game is simple – rock beats scissors (blunts it), scissors beats paper (cuts it), and paper beats rock (wraps it up). Anyone who has ever played this simple game knows that it is not sensible to have any favoured play – your opponent will soon notice this and switch to the winning counter-play. The best strategy (a Nash equilibrium) is to play a mixed random game with any of the three plays taken a third of the time. This, in EGT terms, is a mixed strategy. But many lifeforms are incapable of mixed behavior — they only exhibit one strategy (known as a “pure” strategy”). If the game is played only with the pure Rock, Paper and Scissors strategies the evolutionary game is dynamically unstable: Rock mutants can enter an all scissor population, but then – Paper mutants can take over an all Rock population, but then – Scissor mutants can take over an all Paper population – and on and on…. This is easily seen on the game payoff matrix, where if the paths of mutant invasion are noted, it can be seen that the mutant "invasion paths" form into a loop. This in triggers a cyclic invasion behaviour.

​

                                                                           石头、纸头、剪刀动力学

 

 A computer simulation of the Rock Scissors Paper evolutionary game. The associated RPS Game Payoff Matrix is shown. Starting with an arbitrary population the percentage of the three morphs builds up into a continuously cycling pattern – with rock leading paper, paper leading scissors, and scissors leading rock etc. etc.

​

侧斑蜥蜴

The side-blotched lizard (Uta stansburiana) is polymorphic with three morphs that each pursues a different mating strategy

侧斑蜥蜴（Uta stansburiana）

1) The orange throat is very aggressive and operates over a large territory - attempting to mate with numerous females within this larger area

​

2) The unaggressive yellow throat (called “sneakers”) mimic the markings/behavior of female lizards and sneakily slip into the orange throat's territory to mate with the females there (thereby overtaking the population), and

 

3) The blue throat who mates with and carefully guards ONE female - making it impossible for the sneakers to succeed and therefore overtakes their place in a population…

​

However the blue throats cannot overcome the more aggressive orange throats… And on, and on… The overall situation corresponds to the exact form of the Rock, Scissors, Paper game and the dynamics is much the same. The populations for these lizards actually cycle on a six-year basis. Once again EGT explains a very curious and otherwise inexplicable behaviour in the field of biology. When he read that lizards of the species Uta stansburia were essentially engaged in a game with rock-paper-scissors structure John Maynard Smith exclaimed `They have read my book!

​

RPS（角色扮演策略）和生态

 

Rock Paper Scissors, incorporated into an evolutionary game has been used for modeling natural processes in the study of Ecology. In two-strategy “pairwise” contests, usually one strategy becomes dominant then maintains its dominance, bringing the system to a static ESS point. Interpreted as modelling the competition between species in nature, this suggests that one species will tend to dominate a population of organisms occupying a single ecological niche. However, most ecologies support many species, each dominating its own niche. With more than two species present, mechanisms come into play which bring more complex dynamics to these systems. For example, competition occurring now between two of the species might indirectly help a third by weakening a future rival before direct competition takes place. In such cases, the ESS point can take on a more complex character; even developing a distribution of species which is cyclic over time. The Rock Paper Scissors game was used as the basis for modelling these multi-species ecologies, producing dynamics closely matching observations from nature.

​

RPS（角色扮演策略）和人类社会循环行为

 

Using experimental economics methods, scientists have used RPS game to test the human social evolutionary dynamical behaviors in laboratory. The social cyclic behaviors, predicted by evolutionary game theory, have been also observed in the laboratory experiments with various protocol. These findings mean that evolutionary game theory can be a powerful framework for social science as well as for biology and ecology.

​

发信号、性选择和障碍原则

 

障碍原则在发生作用

 

Aside from the difficulty of explaining how altruism exists in many evolved organisms, Darwin was also bothered by a second conundrum – why do a significant number of species have phenotypical attributes that are patently disadvantageous to them with respect to their survival – and should by the process of natural section be selected against – e.g. the massive inconvenient feather structure found in a peacock's tail? Regarding this issue Darwin wrote to a colleague “The sight of a feather in a peacocks tail, whenever I gaze at it, makes me sick”.

It is the mathematics of evolutionary game theory, which has not only explained the existence of altruism but also explains the totally counterintuitive existence of the peacock’s tail and other such biological encumbrances.

On analysis, problems of biological life are not at all unlike the problems that define economics – eating (akin to resource acquisition and management), survival (competitive strategy) and reproduction (investment, risk and return). Game theory was originally conceived as a mathematical analysis of economic processes and indeed this is why it has proven so useful in explaining so many biological behaviours. One important further refinement of the EGT model that has particular economic overtones rests on the analysis of COSTS. A simple model of cost assumes that all competitors suffer the same penalty imposed by the Game costs, but this is not the case. More successful players will be endowed with or will have accumulated a higher “wealth reserve” or “affordability” than less successful players. This wealth effect in evolutionary game theory is represented mathematically by “resource holding potential  (RHP)” and shows that the effective cost to a competitor with higher RHP are not as great as for a competitor with a lower RHP. As a higher RHP individual is more desirable mate in producing potentially successful offspring, it is only logical that with sexual selection RHP should have evolved to be signalled in some way by the competing rivals, and for this to work this signalling must be done honestly. Amotz Zahavi has developed this thinking in what is known as the Handicap Principle, where superior competitors signal their superiority by a costly display. As higher RHP individuals can properly afford such a costly display this signalling is inherently honest, and can be taken as such by the signal receiver. Nowhere in nature is this better illustrated than in the magnificent and costly plumage of the peacock. The mathematical proof of the handicap principle was developed by Alan Grafen using evolutionary game-theoretic modelling.

​

协同演化

Two types of dynamics have been discussed so far in this article:

• -Evolutionary games which lead to a stable situation or point of stasis for contending strategies which result in an evolutionarily stable strategy

•  
• -Evolutionary games which exhibit a cyclic behaviour (as with RPS game) where the proportions of contending strategies continuously cycle over time within the overall population
• ​

竞争性的协同演化- The rough-skinned newt (Tarricha granulosa) has enough poison in its body to kill a human being 30 times over. This excess of lethal capability is due to the fact that it is involved in an arms race with a specific predator, the common garter snake (Thamnophis sirtalis), which in response to the newt's poisonous defenses has over time evolved to be highly tolerant of the poison. The two species are thereby locked in a "Red Queen" arms race.

双赢协同演化

- Darwin's orchid (Angraecum sesquipedale) and the moth Morgan's sphinx (Xanthopan morgani) like many insect and flower "partners" have a mutual relationship where the moth gains pollen and the flower pollination. To insure efficiency in this particular exchange the pair have evolved a mechanism which excludes the waste of pollen transfer to/from another flower species and assures feeding pollen only to the "proper" pollinator. The mechanism is an extraordinarily long proboscis on the moth and a equally long nectary on the orchid.

A third dynamic can exist in certain more complex systems, where combinations of effects are in play, which contain not only intra-species competition but also inter-species competition as well. This is the realm of co-evolution. Examples include predator-prey competition and host-parasite competition. There are also co-evolutionary situations possible where two species interact where mutual interests are at play in the dynamic. Evolutionary game models have been created to encompass both these co-evolutionary situations for pairwise and multi-species coevolutionary systems.[43] The general dynamic differs between competitive systems and mutualistic systems.

In competitive (non-mutualistic) inter-species coevolutionary system the species are involved in an “arms race” - where adaptions that are better at competing against the other species are less likely to be culled. Both game payoffs and replicator dynamics are thereby affected to reflect this. The “counterstrategy” of the other competitor species is similarly affected, exposing them to a positive selection pressure for any effective counter-strategy – in effect creating an overall competitive arms race. Underlying this total situation is an ecological effect in which the population dynamic of the competing species itself influences outcomes- e.g. a mutant strategy in a predator population which enables that species to drive its sole prey species to extinction is itself doomed. This all leads in effect to a “Red Queen”dynamic where, as in Alice in Wonderland, the protagonists must “run as fast as they can to just stay in one place”.

A number of EGT models have been produced to encompass coevolutionary situations. Modelling these multi-component coevolutionary systems mathematically is necessarily complex. This is particularly true, as a key factor applicable in these coevolutionary systems is the continuous adaption of strategy taking place in the dynamics of such arms races. Most other evolutionary games, e.g. resource conflicts, do not need to include this process of dynamic mutation in the analysis of game dynamics itself. Therefore, coevolutionary modelling often involves a cross-over area, where both EGT and Genetic Algorithms reflecting mutational effects are used in the models, often having computers simulate the dynamics of the overall coevolutionary game. The resulting dynamics are studied as various parameters are modified. Because a multitude of variables are simultaneously at play in these complex dynamics, solutions become the province of multi-variable optimality, rather than simply a single ESS. The mathematical criteria of determining stable points are Pareto efficiency and Pareto dominance, which is a measure of solution optimality peaks in these interdependent multivariable systems.

In a paper by Carl Bergstrom and Michael Lachmann, the authors successfully apply evolutionary game theory models to understand the division of benefits in mutualistic interactions between organisms. Darwinian assumptions about fitness are modeled using replicator dynamics to show that the organism evolving at a slower rate in a mutualistic relationship will gain a disproportionately high share of the benefits or payoffs. This application of EGT provided an interesting and perhaps unexpected twist on the Red Queen Hypothesis which concludes evolution favored faster rates of evolution.

​

演化博弈论的延伸

A mathematical model analysing the behaviour of a system needs initially to be as simple as possible to aid in developing a base understanding the fundamentals, or “first order effects”, pertaining to what is being studied. With this understanding in place it is then appropriate to see if other, more subtle, parameters (second order effects) further impact the primary behaviours or shape additional behaviours in the system. Following Maynard Smith’s seminal work in EGT, the subject has had a number of very significant extensions which have shed more light on understanding evolutionary dynamics, particularly in the area of altruistic behaviors. Some of these key extensions to EGC are:

空间博弈： In a spatial evolutionary game contestants meet in contests at fixed grid positions and only interact with immediate neighbors. Shown here are the dynamics of a Hawk Dove contest, showing Hawk and Dove contestants as well as the changes of strategy taking place in the various cells

 

一、空间博弈：There are several very important influences attributable to geographic factors in the field of evolution – not least of which is gene flow and horizontal gene transfer. Spatial game models take geometry into effect by locating contestants in a lattice of cells over a two dimensional plane to represent this geographic element. In spatial games contests take place only with immediate neighbours. Winning strategies take over these immediate neighbourhoods and then go on to interact further with adjacent neighbourhoods. This model has been especially useful in showing how pockets of co-operators can invade and introduce altruism in the Prisoners Dilemma game, where for example Tit for Tat (TFT) is a Nash Equilibrium but NOT also an ESS. TFT cannot normally invade an all Defector population yet in spatial games this becomes possible. The basic principle of spatial structure is sometimes abstracted into a general network of interactions. This is the foundation of evolutionary graph theory.

​二、拥有信息的效果： In conventional Game Theory the effect of Signalling (the acquisition of information) is of critical importance in outcomes. This can also be the case in EGT. Nowhere is this more important than in the area of studying Indirect Reciprocity in Prisoners Dilemma (where contests between the SAME paired individuals is NOT repetitive). This models the reality of most normal social interactions which are non-kin related. It has been shown that unless a probability measure of REPUTATION is available in Prisoners Dilemma only DIRECT reciprocity can be achievable.[26] With this information INDIRECT reciprocity is also supported.  Alternatively, agents might have access to an arbitrary signal that is initially uncorrelated to strategy but becomes correlated due to evolutionary dynamics. This is known as the green-beard effect or evolution of ethnocentrism in humans.[50] Depending on the game, It can allow the evolution of either cooperation or irrational hostility. More generally, there are many instances in biology from the molecular to the multicellular levels where a signaling game model, with information asymmetry between sender and receiver, might be appropriate, such as in mate attraction or evolution of translation machinery from RNA strings.

三、有限种群：Many evolutionary games have been modelled in finite populations rather than infinite to see the effect this may have. Though in many cases this does not significantly alter game dynamics in others significant differences can and do occur, for example being in the prevalence of mixed strategies.

​

其他应用

The successful application of game theory to evolution has brought further insights to human behavior. Whereas game theory traditionally assumes rational actors, in the real world this does not always describe human behavior. EGT has predicted behaviors in animals where strong assumptions of rationality

 cannot be made. An important feature of all these sets of models under the umbrella of evolutionary game theory is repetition. If the games were not repeated, these EGT models would not be able to provide any insight into adaptive behaviors and strategies due to the dynamic nature of the mechanisms of evolution. Further, this biological application is meaningful for economics because it provides an understanding of the adjustments that occur between two equilibria (Samuelson 2002). While game theory provides a framework within which biologists can learn and understand organisms, the observation of evolution and how these strategies are applied helps economics illuminate processes. It is interesting to note however, that Maynard Smith himself never felt that the use of evolutionary game theory in the field of economics was quite relevant.[53] In a new introduction for his book "The Alchemy of Finance", George Soros says: "While I still cannot claim full familiarity with the literature, I see some merit in behavioral economics and even more merit in evolutionary game theory. I consider the latter a practical exploration of reflexivity." Moreover,the game theory models of the pathogenic microbes may provide us with advanced therapeutic ways.