The Law of Large Numbers

Flip a fair coin twice. You might see two heads in a row — that's a 50 % rate, exactly right. You might see two tails — a 0 % rate, embarrassingly wrong. Short runs can be wildly misleading.

Flip a fair coin a million times and something remarkable happens: the fraction of heads will be so close to 0.5 that the difference is negligible for almost any practical purpose. You can bet your savings on it, and mathematicians will back you up.

That guarantee is called the Law of Large Numbers (LLN). It says that the sample mean of independent, identically distributed observations will converge to the expected value as the number of observations grows. It is one of the few certainties in a discipline built around uncertainty, and it is the mathematical bedrock under casinos, polling, insurance, and machine-learning algorithms alike.

The result was stated precisely by Jakob Bernoulli in 1713 in his posthumous Ars Conjectandi — the first rigorous proof that probability had any connection to long-run frequency. Before Bernoulli, "likely" was just a vague feeling. After him, it was a theorem.

Press Flip coins to toss a batch of fair coins and plot the running average of heads. Watch how the curve swings wildly at first, then tightens relentlessly toward 0.5 as the total count grows.

<p class="hint">{{hint}}</p>
<canvas id="chart" width="520" height="220"></canvas>
<div class="stats" id="stats">{{stats_initial}}</div>
<div class="btns">
  <button id="btn10"  type="button">{{btn10}}</button>
  <button id="btn100" type="button">{{btn100}}</button>
  <button id="btn500" type="button">{{btn500}}</button>
  <button id="reset"  type="button" class="ghost">{{reset}}</button>
</div>

* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; color: #222; margin: 0; background: #fff; }
.hint { font-size: .88rem; color: #444; margin: 0 0 .6rem; line-height: 1.45; }
canvas { display: block; width: 100%; max-width: 520px; border: 1px solid #dde3ea;
         border-radius: 10px; background: #f7f9fb; }
.stats { font-size: .92rem; margin: .5rem 0; color: #333; min-height: 1.4em; }
.btns { display: flex; gap: .5rem; flex-wrap: wrap; margin-top: .4rem; }
button { font: 600 14px system-ui, sans-serif; padding: .42rem .85rem;
         border: 1px solid #1d3557; background: #1d3557; color: #fff;
         border-radius: 8px; cursor: pointer; }
button.ghost { background: #fff; color: #1d3557; }

// Code not found

Notice that early flips have enormous power — a string of heads at the start bends the curve far from 0.5. But every additional batch of flips shrinks the deviation. The band around 0.5 narrows like a funnel. By 500 flips the average rarely strays beyond ±0.05; by 5000 it is within ±0.02. The LLN does not say the count of heads exactly equals the count of tails — it says the proportion becomes indistinguishable from 0.5.

There are actually two versions of the Law of Large Numbers, and the distinction matters:

Weak LLN (Chebyshev / Poisson, ~1867) For any $\epsilon > 0$, the probability that the sample mean deviates from $\mu$ by more than $\epsilon$ goes to zero as $n \to \infty$. Formally:

$$P(|\bar{X}_n - \mu| > \epsilon) \to 0 \text{ as } n \to \infty$$

This says: for any fixed tolerance, the chance of exceeding it vanishes. But it does not rule out an infinite sequence where deviations keep happening — just ever more rarely.

Strong LLN (Kolmogorov, 1933) The sample mean converges to $\mu$ almost surely — that is, with probability 1, the sequence $\bar{X}_1, \bar{X}_2, \bar{X}_3, \ldots$ eventually stays within $\epsilon$ of $\mu$ forever. Formally:

$$P\!\left(\lim_{n \to \infty} \bar{X}_n = \mu\right) = 1$$

This is the stronger promise: not just that deviations get unlikely, but that almost every possible infinite sequence of flips eventually and permanently settles at $\mu$.

What makes it work: the proof of the weak LLN is a single application of Chebyshev's inequality — if a random variable has finite variance $\sigma^2$, then $P(|\bar{X}_n - \mu| > \epsilon) \le \sigma^2 / (n\epsilon^2)$. As $n$ grows, this bound goes to zero. The strong LLN requires more care (Borel–Cantelli lemma), but the key ingredient is the same: variance shrinks as $1/n$.

Status: both theorems are proven — fully rigorous mathematical facts, not empirical observations. No amount of computer simulation is needed to establish them; the proofs in any graduate probability textbook suffice. This contrasts sharply with open questions like P vs NP or the Riemann Hypothesis, where the truth is still unknown.

The gambler's fallacy lives in the gap between proportion and count. The LLN says the fraction of heads approaches 0.5; it says nothing about the absolute difference between heads and tails — which actually tends to grow like $\sqrt{n}$. Thinking "I'm due for a head" because there have been many tails is a misreading of the theorem.

The LLN is the silent engine behind dozens of industries and algorithms:

• Casinos: every game has a house edge — a positive expected profit per bet. The LLN guarantees that over thousands of bets, the casino's actual profit converges to that edge. A lucky player can beat the house tonight; the house always beats the population of players over a full year.
• Insurance: a single car accident is unpredictable. The average claim rate across a million policies is not. Actuaries price policies using expected claim costs, and the LLN ensures the portfolio tracks that expectation.
• Polling: a random sample of 1,000 voters has roughly ±3 % margin of error. This flows directly from the LLN (and the related Central Limit Theorem): the sample proportion converges to the population proportion.
• Monte Carlo methods: need the area under a curve? Sample random points and count how many land under it. The LLN guarantees the fraction converges to the true area — no closed-form integral needed.
• Machine learning: stochastic gradient descent uses a random mini-batch to estimate the full-dataset gradient. The LLN justifies why an average over a small batch is a reliable proxy for the true gradient.
• Quality control: a factory tests a sample of widgets. The defect rate in the sample converges to the true process defect rate — enabling statistically valid decisions without testing every unit.

The Law of Large Numbers is one of mathematics' most reassuring theorems: randomness is wild in the short run, but it is tame in the long run. No matter how erratic individual coin flips are, their average will hug 0.5 as surely as the sun rises.

That guarantee is not intuition or folklore — it is a proven mathematical fact, first stated rigorously by Jakob Bernoulli in 1713 and refined by Poisson, Chebyshev, and Kolmogorov over the following two centuries. The weak law gives you convergence in probability; the strong law gives you almost-sure convergence. Both are theorems, not guesses.

The next time a casino chips away at your stack, or an insurance premium feels too high, or a political poll claims a margin of error — the Law of Large Numbers is the machinery quietly running in the background, making the unpredictable predictable. See also the companion result, the Central Limit Theorem, which tells you not just that the average converges, but how fast and what shape the distribution takes along the way.