Introduction

Flip a fair coin twice. You might see two heads in a row — that's a 50 % rate, exactly right. You might see two tails — a 0 % rate, embarrassingly wrong. Short runs can be wildly misleading.

Flip a fair coin a million times and something remarkable happens: the fraction of heads will be so close to 0.5 that the difference is negligible for almost any practical purpose. You can bet your savings on it, and mathematicians will back you up.

That guarantee is called the Law of Large Numbers (LLN). It says that the sample mean of independent, identically distributed observations will converge to the expected value as the number of observations grows. It is one of the few certainties in a discipline built around uncertainty, and it is the mathematical bedrock under casinos, polling, insurance, and machine-learning algorithms alike.

The result was stated precisely by Jakob Bernoulli in 1713 in his posthumous Ars Conjectandi — the first rigorous proof that probability had any connection to long-run frequency. Before Bernoulli, "likely" was just a vague feeling. After him, it was a theorem.

Watch It Converge

Press Flip coins to toss a batch of fair coins and plot the running average of heads. Watch how the curve swings wildly at first, then tightens relentlessly toward 0.5 as the total count grows.

<p class="hint">Flip batches of fair coins and watch the running proportion of heads
converge toward <strong>0.5</strong>. The gray band shows ±1 standard deviation
(σ = √(p(1−p)/n)). Early flips swing wildly; the band tightens with every batch.</p>
<canvas id="chart" width="520" height="220"></canvas>
<div class="stats" id="stats">Flips: 0 &nbsp;|&nbsp; Heads: 0 &nbsp;|&nbsp; Proportion: —</div>
<div class="btns">
  <button id="btn10"  type="button">+10 flips</button>
  <button id="btn100" type="button">+100 flips</button>
  <button id="btn500" type="button">+500 flips</button>
  <button id="reset"  type="button" class="ghost">Reset</button>
</div>

* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; color: #222; margin: 0; background: #fff; }
.hint { font-size: .88rem; color: #444; margin: 0 0 .6rem; line-height: 1.45; }
canvas { display: block; width: 100%; max-width: 520px; border: 1px solid #dde3ea;
         border-radius: 10px; background: #f7f9fb; }
.stats { font-size: .92rem; margin: .5rem 0; color: #333; min-height: 1.4em; }
.btns { display: flex; gap: .5rem; flex-wrap: wrap; margin-top: .4rem; }
button { font: 600 14px system-ui, sans-serif; padding: .42rem .85rem;
         border: 1px solid #1d3557; background: #1d3557; color: #fff;
         border-radius: 8px; cursor: pointer; }
button.ghost { background: #fff; color: #1d3557; }

const fmt = (s, v) => s.replace(/%(\w+)%/g, (_, k) => String(v[k] ?? ''));
// Law of Large Numbers demo: flip fair coins, track running proportion of heads.
// After n flips the proportion X̄ₙ = (# heads) / n.
// LLN guarantees X̄ₙ → 0.5 (the expected value) as n → ∞.
// The gray band is ±1 SD = ±√(0.25/n) — shrinks like 1/√n.

const canvas = document.getElementById('chart');
const ctx = canvas.getContext('2d');
const W = canvas.width, H = canvas.height;
const PAD = { top: 18, right: 16, bottom: 28, left: 42 };
const PW = W - PAD.left - PAD.right;
const PH = H - PAD.top  - PAD.bottom;

let totalFlips = 0, totalHeads = 0;
// Store (n, proportion) samples to plot
const samples = []; // {n, p}

function seeded() {
  // fast xorshift32 — deterministic enough for a demo
  // We just use Math.random for simplicity
  return Math.random() < 0.5 ? 1 : 0;
}

function flip(n) {
  for (let i = 0; i < n; i++) {
    totalFlips++;
    totalHeads += seeded();
    // Record every flip up to 50, then thin out
    if (totalFlips <= 50 || totalFlips % Math.ceil(totalFlips / 200) === 0) {
      samples.push({ n: totalFlips, p: totalHeads / totalFlips });
    }
  }
  // Always record the final point
  samples.push({ n: totalFlips, p: totalHeads / totalFlips });
  draw();
  updateStats();
}

function updateStats() {
  const p = totalFlips > 0 ? (totalHeads / totalFlips).toFixed(4) : '—';
  document.getElementById('stats').innerHTML =
    fmt('Flips: <b>%totalFlips%</b> &nbsp;|&nbsp; Heads: <b>%totalHeads%</b> &nbsp;|&nbsp; Proportion: <b>%p%</b>', { totalFlips: totalFlips, totalHeads: totalHeads, p: p });
}

function xFor(n) {
  if (samples.length < 2) return PAD.left;
  const maxN = samples[samples.length - 1].n;
  return PAD.left + (n / maxN) * PW;
}

function yFor(p) {
  // p in [0,1], map to canvas y (0=top, H=bottom)
  // show range [0,1]
  return PAD.top + PH * (1 - p);
}

function draw() {
  ctx.clearRect(0, 0, W, H);
  if (samples.length === 0) { drawAxes(); return; }

  const maxN = samples[samples.length - 1].n;

  // --- SD band ---
  ctx.beginPath();
  // upper band
  for (let i = 0; i < samples.length; i++) {
    const s = samples[i];
    const sd = Math.sqrt(0.25 / s.n);
    const x = PAD.left + (s.n / maxN) * PW;
    const y = yFor(Math.min(1, 0.5 + sd));
    i === 0 ? ctx.moveTo(x, y) : ctx.lineTo(x, y);
  }
  // lower band (reverse)
  for (let i = samples.length - 1; i >= 0; i--) {
    const s = samples[i];
    const sd = Math.sqrt(0.25 / s.n);
    const x = PAD.left + (s.n / maxN) * PW;
    const y = yFor(Math.max(0, 0.5 - sd));
    ctx.lineTo(x, y);
  }
  ctx.closePath();
  ctx.fillStyle = 'rgba(100,160,220,0.15)';
  ctx.fill();

  // --- 0.5 reference line ---
  ctx.beginPath();
  ctx.moveTo(PAD.left, yFor(0.5));
  ctx.lineTo(PAD.left + PW, yFor(0.5));
  ctx.strokeStyle = '#e63946';
  ctx.lineWidth = 1.2;
  ctx.setLineDash([5, 4]);
  ctx.stroke();
  ctx.setLineDash([]);

  // --- proportion curve ---
  ctx.beginPath();
  for (let i = 0; i < samples.length; i++) {
    const s = samples[i];
    const x = PAD.left + (s.n / maxN) * PW;
    const y = yFor(s.p);
    i === 0 ? ctx.moveTo(x, y) : ctx.lineTo(x, y);
  }
  ctx.strokeStyle = '#1d3557';
  ctx.lineWidth = 1.8;
  ctx.stroke();

  drawAxes(maxN);

  // label 0.5 line
  ctx.fillStyle = '#e63946';
  ctx.font = '11px system-ui, sans-serif';
  ctx.fillText('0.5', PAD.left + PW + 2, yFor(0.5) + 4);
}

function drawAxes(maxN) {
  ctx.strokeStyle = '#aab';
  ctx.lineWidth = 1;

  // y-axis
  ctx.beginPath(); ctx.moveTo(PAD.left, PAD.top); ctx.lineTo(PAD.left, PAD.top + PH); ctx.stroke();
  // x-axis
  ctx.beginPath(); ctx.moveTo(PAD.left, PAD.top + PH); ctx.lineTo(PAD.left + PW, PAD.top + PH); ctx.stroke();

  ctx.fillStyle = '#555';
  ctx.font = '10px system-ui, sans-serif';
  // y labels
  [0, 0.25, 0.5, 0.75, 1].forEach(p => {
    const y = yFor(p);
    ctx.fillText(p.toFixed(2), 2, y + 3);
    ctx.beginPath(); ctx.moveTo(PAD.left - 3, y); ctx.lineTo(PAD.left, y); ctx.stroke();
  });

  // x label
  if (maxN) {
    ctx.fillText('n=' + maxN, PAD.left + PW - 20, PAD.top + PH + 16);
    ctx.fillText('n=1', PAD.left, PAD.top + PH + 16);
  }

  // axis title
  ctx.save();
  ctx.translate(10, PAD.top + PH / 2);
  ctx.rotate(-Math.PI / 2);
  ctx.fillText('proportion', -28, 0);
  ctx.restore();
}

document.getElementById('btn10').addEventListener('click', () => flip(10));
document.getElementById('btn100').addEventListener('click', () => flip(100));
document.getElementById('btn500').addEventListener('click', () => flip(500));
document.getElementById('reset').addEventListener('click', () => {
  totalFlips = 0; totalHeads = 0; samples.length = 0;
  ctx.clearRect(0, 0, W, H); drawAxes(); updateStats();
});

// Draw empty axes on load
drawAxes();
updateStats();

// Quick self-test (runs silently in the console)
(function selfTest() {
  // Simulate 10000 fair coin flips and check proportion within 1%
  let h = 0, n = 10000;
  // Use a deterministic PRNG (xorshift32) so the test is reproducible
  let seed = 123456789;
  function rand() { seed ^= seed << 13; seed ^= seed >> 17; seed ^= seed << 5; return (seed >>> 0) / 4294967296; }
  for (let i = 0; i < n; i++) h += rand() < 0.5 ? 1 : 0;
  const prop = h / n;
  const ok = Math.abs(prop - 0.5) < 0.02;
  console.log(fmt('[LLN self-test] n=%n% proportion=%prop% ok=%ok%', { n: n, prop: prop.toFixed(4), ok: ok }));
  if (!ok) console.error('[LLN self-test] FAILED — proportion too far from 0.5');
})();
Result
Console

Notice that early flips have enormous power — a string of heads at the start bends the curve far from 0.5. But every additional batch of flips shrinks the deviation. The band around 0.5 narrows like a funnel. By 500 flips the average rarely strays beyond ±0.05; by 5000 it is within ±0.02. The LLN does not say the count of heads exactly equals the count of tails — it says the proportion becomes indistinguishable from 0.5.

The Real Theorem

There are actually two versions of the Law of Large Numbers, and the distinction matters:

Weak LLN (Chebyshev / Poisson, ~1867) For any ϵ>0\epsilon > 0, the probability that the sample mean deviates from μ\mu by more than ϵ\epsilon goes to zero as n→∞n \to \infty. Formally:

P(∣Xˉn−μ∣>ϵ)→0 as n→∞P(|\bar{X}_n - \mu| > \epsilon) \to 0 \text{ as } n \to \infty

This says: for any fixed tolerance, the chance of exceeding it vanishes. But it does not rule out an infinite sequence where deviations keep happening — just ever more rarely.

Strong LLN (Kolmogorov, 1933) The sample mean converges to μ\mu almost surely — that is, with probability 1, the sequence Xˉ1,Xˉ2,Xˉ3,…\bar{X}_1, \bar{X}_2, \bar{X}_3, \ldots eventually stays within ϵ\epsilon of μ\mu forever. Formally:

P ⁣(lim⁡n→∞Xˉn=μ)=1P\!\left(\lim_{n \to \infty} \bar{X}_n = \mu\right) = 1

This is the stronger promise: not just that deviations get unlikely, but that almost every possible infinite sequence of flips eventually and permanently settles at μ\mu.

What makes it work: the proof of the weak LLN is a single application of Chebyshev's inequality — if a random variable has finite variance σ2\sigma^2, then P(∣Xˉn−μ∣>ϵ)≤σ2/(nϵ2)P(|\bar{X}_n - \mu| > \epsilon) \le \sigma^2 / (n\epsilon^2). As nn grows, this bound goes to zero. The strong LLN requires more care (Borel–Cantelli lemma), but the key ingredient is the same: variance shrinks as 1/n1/n.

Status: both theorems are proven — fully rigorous mathematical facts, not empirical observations. No amount of computer simulation is needed to establish them; the proofs in any graduate probability textbook suffice. This contrasts sharply with open questions like P vs NP or the Riemann Hypothesis, where the truth is still unknown.

The gambler's fallacy lives in the gap between proportion and count. The LLN says the fraction of heads approaches 0.5; it says nothing about the absolute difference between heads and tails — which actually tends to grow like n\sqrt{n}. Thinking "I'm due for a head" because there have been many tails is a misreading of the theorem.

Where It Matters

The LLN is the silent engine behind dozens of industries and algorithms:

  • Casinos: every game has a house edge — a positive expected profit per bet. The LLN guarantees that over thousands of bets, the casino's actual profit converges to that edge. A lucky player can beat the house tonight; the house always beats the population of players over a full year.
  • Insurance: a single car accident is unpredictable. The average claim rate across a million policies is not. Actuaries price policies using expected claim costs, and the LLN ensures the portfolio tracks that expectation.
  • Polling: a random sample of 1,000 voters has roughly ±3 % margin of error. This flows directly from the LLN (and the related Central Limit Theorem): the sample proportion converges to the population proportion.
  • Monte Carlo methods: need the area under a curve? Sample random points and count how many land under it. The LLN guarantees the fraction converges to the true area — no closed-form integral needed.
  • Machine learning: stochastic gradient descent uses a random mini-batch to estimate the full-dataset gradient. The LLN justifies why an average over a small batch is a reliable proxy for the true gradient.
  • Quality control: a factory tests a sample of widgets. The defect rate in the sample converges to the true process defect rate — enabling statistically valid decisions without testing every unit.

Conclusion

The Law of Large Numbers is one of mathematics' most reassuring theorems: randomness is wild in the short run, but it is tame in the long run. No matter how erratic individual coin flips are, their average will hug 0.5 as surely as the sun rises.

That guarantee is not intuition or folklore — it is a proven mathematical fact, first stated rigorously by Jakob Bernoulli in 1713 and refined by Poisson, Chebyshev, and Kolmogorov over the following two centuries. The weak law gives you convergence in probability; the strong law gives you almost-sure convergence. Both are theorems, not guesses.

The next time a casino chips away at your stack, or an insurance premium feels too high, or a political poll claims a margin of error — the Law of Large Numbers is the machinery quietly running in the background, making the unpredictable predictable. See also the companion result, the Central Limit Theorem, which tells you not just that the average converges, but how fast and what shape the distribution takes along the way.

Share this article

Pick a channel — or use your device's native share sheet.

Comments

Loading comments...

https://www.kipuhub.com/en/article/law-of-large-numbers/Content licensed under CC BY-NC 4.0.