Evolutionarily Stable Strategies

Animals fight. But if you watch a pair of stags locking antlers, you notice something odd: they rarely kill each other. They push, display, and one retreats. Why? Raw fitness logic says the stronger animal should press every advantage. So why hold back?

In 1973 the biologist John Maynard Smith and mathematician George R. Price asked exactly that question and answered it with game theory. Their paper "The Logic of Animal Conflict" introduced the evolutionarily stable strategy (ESS): a behavioural strategy so robust that, once a population adopts it, no rare mutant with a different strategy can invade and spread.

An ESS is not quite a Nash equilibrium — it is stronger. It demands that, when the mutant is rare, every individual playing the resident strategy does better than the mutant does. Natural selection then weeds the mutant out, and the population returns to the ESS. The result is evolution's version of a locked-in equilibrium.

The simplest model is the Hawk-Dove game. Imagine a resource worth V fitness points. Hawks always escalate; Doves always display and retreat if the opponent escalates. The cost of an injury from a real fight is C. When V < C, neither a fully hawk nor a fully dove population is an ESS — the stable outcome is a mixed equilibrium in which each individual plays Hawk with probability V/C. That fraction is, famously, exactly the proportion of hawks you would observe in a real population at steady state.

Adjust the resource value V and injury cost C, then press Run to watch a population of 200 individuals evolve over generations. Each round every individual is paired at random with another; payoffs accumulate; low-fitness individuals are replaced by offspring of high-fitness ones (proportional selection with mutation).

<div class="controls">
  <label>{{lbl_resource_v}} <span id="vval">4</span>
    <input type="range" id="vslider" min="1" max="10" value="4" step="1">
  </label>
  <label>{{lbl_injury_c}} <span id="cval">10</span>
    <input type="range" id="cslider" min="2" max="20" value="10" step="1">
  </label>
  <div class="ess-display">{{lbl_ess_freq}} <span id="essfreq">0.400</span></div>
</div>
<canvas id="chart" width="480" height="200"></canvas>
<div class="pop-bar">
  <div id="hawk-bar" class="hawk-seg"></div>
  <div id="dove-bar" class="dove-seg"></div>
</div>
<div class="pop-labels">
  <span class="hawk-label">{{lbl_hawks}} <span id="hawk-pct">50%</span></span>
  <span class="dove-label">{{lbl_doves}} <span id="dove-pct">50%</span></span>
</div>
<div class="btns">
  <button id="run" type="button">{{btn_run}}</button>
  <button id="step" type="button">{{btn_step}}</button>
  <button id="reset" type="button" class="ghost">{{btn_reset}}</button>
</div>
<div class="status" id="status">{{status_initial}}</div>

* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; color: #222; margin: 0; padding: 14px 4px; }
.controls { display: flex; flex-direction: column; gap: .4rem; margin-bottom: .6rem; font-size: .9rem; }
.controls label { display: flex; align-items: center; gap: .5rem; }
.controls input[type=range] { flex: 1; }
.ess-display { font-weight: 700; color: #1d3557; font-size: .9rem; }
canvas { display: block; width: 100%; max-width: 480px; border: 1px solid #cdd9e3; border-radius: 8px; background: #f7f9fb; }
.pop-bar { display: flex; height: 22px; border-radius: 6px; overflow: hidden; margin: .5rem 0 .2rem; border: 1px solid #cdd9e3; }
.hawk-seg { background: #e63946; transition: width .2s; }
.dove-seg { background: #457b9d; flex: 1; transition: width .2s; }
.pop-labels { display: flex; justify-content: space-between; font-size: .85rem; font-weight: 600; margin-bottom: .4rem; }
.hawk-label { color: #e63946; }
.dove-label { color: #457b9d; }
.btns { display: flex; gap: .5rem; flex-wrap: wrap; margin-top: .4rem; }
button { font: 600 14px system-ui; padding: .4rem .85rem; border: 1px solid #1d3557;
         background: #1d3557; color: #fff; border-radius: 8px; cursor: pointer; }
button.ghost { background: #fff; color: #1d3557; }
.status { font-size: .9rem; margin-top: .4rem; min-height: 1.3em; color: #555; }

// Code not found

The red line shows the fraction of Hawks; the dashed line marks the theoretical ESS frequency V/C. Notice that no matter where you start — all Hawks, all Doves, or anything in between — the population converges to the same equilibrium. That convergence is what makes it stable.

Formal definition

Given a symmetric two-player game with payoff function f(s, r) — the fitness of a player using strategy s against a player using strategy r — a strategy s* is an ESS if and only if, for every alternative strategy t ≠ s*:

1. f(s*, s*) > f(t, s*) (s beats every invader when rare)*, OR
2. f(s*, s*) = f(t, s*) AND f(s*, t) > f(t, t) (tie at resident frequency but s wins in the invader's own environment)*.

This is strictly stronger than Nash: every ESS is a Nash equilibrium, but not every Nash equilibrium is an ESS.

Hawk-Dove: solved analytically (1973)

With V < C, neither pure strategy is an ESS:

• All-Hawk population: a Dove mutant earns V/2 while a Hawk earns (V-C)/2 < V/2 when C > 0. Dove can invade.
• All-Dove population: a Hawk mutant earns V while a Dove earns V/2. Hawk can invade.
• Mixed ESS at frequency p* = V/C: at this frequency, Hawks and Doves earn the same expected payoff, and neither can invade. Proven by Maynard Smith and Price in 1973.

Replicator dynamics

In large populations the proportion p of Hawks changes each generation by the replicator equation:

dp/dt = p(1-p)[f(Hawk, p) - f(Dove, p)]

where f(Hawk, p) = p·(V-C)/2 + (1-p)·V and f(Dove, p) = (1-p)·V/2. The unique interior fixed point p* = V/C is globally stable for V < C — every starting frequency converges to it.

Complexity of finding ESS in general games

For two-strategy games the ESS (if it exists) can be computed in $O(1)$. For multi-strategy games with n pure strategies, deciding whether a given strategy profile is an ESS requires checking $O(n^{2})$ conditions — tractable. However, deciding whether any ESS exists, or finding an ESS among exponentially many mixed strategies in large population games, can be computationally demanding. The related problem of finding Nash equilibria in general-sum games is PPAD-complete, but ESS computation has its own subtleties depending on game structure. For the biologically important class of finite symmetric games, ESS existence is decidable in polynomial time.

Status

The ESS concept is mathematically settled for the classical two-player symmetric case. Extensions to multi-player, multi-locus, and spatial models remain active research areas.

The ESS framework spread far beyond biology after 1973:

• Animal behaviour: ritualized combat, territory displays, parental investment, and the sex ratio are all predicted by ESS analysis. The 1:1 sex ratio (Fisher's principle, independently recovered via ESS) is the clearest empirical confirmation.
• Evolutionary economics: firms in a market evolve strategies; if no firm can profitably deviate, the market is at an ESS. This framing underlies evolutionary game theory in economics.
• Immunology: the arms race between pathogens and immune systems can be modelled as a co-evolutionary game in which each side seeks its ESS against a moving target.
• Cooperation and altruism: Hamilton's kin selection and Axelrod's computer tournaments both ask when cooperative strategies are evolutionarily stable against defectors — the same mathematical question as ESS.
• Auction theory and mechanism design: understanding which bidding strategies are stable against deviation draws on the same equilibrium-stability thinking.

The connection to Nash equilibria is foundational: ESS refines Nash by adding an evolutionary stability condition, giving game theory a dynamic, Darwinian interpretation. See also game theory for how equilibria arise in strategic settings.

The stag that retreats rather than fighting to the death is not being irrational — it is playing an evolutionarily stable strategy. Maynard Smith and Price proved in 1973 that natural selection, repeated over generations, converges to behavioural equilibria that no rare mutant can crack. In the hawk-dove game, that equilibrium is exactly V/C — a precise, testable prediction that has been confirmed across species from insects to deer.

The deeper lesson is that evolution is a game, and the strategies it locks in are the ones that are self-reinforcing under the pressure of invasion. That insight, once stated, turned out to be one of the most productive ideas in twentieth-century biology — and a beautiful example of mathematics illuminating a question that looked purely biological.

To go further: ESS is a refinement of the Nash equilibrium that adds a Darwinian stability criterion. The two concepts together are the foundation of modern evolutionary game theory.