The Central Limit Theorem

Roll a single unfair die and the result can be almost anything. Roll it a hundred times and add up the results — and something remarkable happens: the sum almost certainly lands near the same value, and the variation around that value has a bell-shaped curve.

The shape of the die no longer matters. The die can be six-sided or twenty-sided, fair or wildly biased. As long as the rolls are independent and have a finite variance, their sum converges to the same universal shape: the normal distribution.

This is the Central Limit Theorem (CLT), one of the most important results in all of mathematics. Proven in its modern form by Pierre-Simon Laplace around 1810 (building on earlier work by Abraham de Moivre in 1733), it explains why the bell curve appears in phenomena as different as measurement errors, human heights, test scores, and stock-price fluctuations — even when the underlying individual events are anything but bell-shaped.

The theorem does not say randomness disappears when you add more variables. It says randomness takes a predictable shape — and that shape is always the same.

The die below is lopsided: it lands on 6 much more often than on 1. Roll it once and the result is unpredictable. But sum many rolls together — and watch what the histogram of those sums does.

<div class="controls">
  <label>{{lbl_dice_per_roll}}: <strong id="nlabel">1</strong>
    <input type="range" id="nslider" min="1" max="20" value="1">
  </label>
  <label>{{lbl_rolls}}: <strong id="rlabel">500</strong>
    <input type="range" id="rslider" min="100" max="2000" step="100" value="500">
  </label>
  <button id="rollbtn" type="button">{{btn_roll}}</button>
</div>
<div class="die-strip">
  <span class="die-label">{{lbl_die_probs}}</span>
  <div id="diebars"></div>
</div>
<canvas id="chart" width="480" height="260"></canvas>
<p id="statsline" class="stats"></p>

* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; margin: 0; color: #222; }
.controls { display: flex; flex-wrap: wrap; gap: .7rem 1.2rem; align-items: center; margin-bottom: .6rem; }
label { font-size: .88rem; display: flex; align-items: center; gap: .4rem; }
input[type=range] { width: 110px; accent-color: #1d3557; }
button { font: 600 14px system-ui; padding: .4rem .9rem; background: #1d3557; color: #fff;
         border: none; border-radius: 8px; cursor: pointer; }
button:hover { background: #274472; }
.die-strip { margin-bottom: .5rem; }
.die-label { font-size: .8rem; color: #555; }
#diebars { display: flex; gap: 3px; align-items: flex-end; height: 40px; margin-top: 4px; }
.diebar { width: 28px; display: flex; flex-direction: column; align-items: center; gap: 2px; }
.diebar-fill { width: 100%; background: #457b9d; border-radius: 3px 3px 0 0; transition: height .3s; }
.diebar-lbl { font-size: 9px; color: #444; }
canvas { display: block; max-width: 100%; border: 1px solid #dde3ea; border-radius: 8px; background: #f8fafc; }
.stats { font-size: .85rem; color: #555; margin: .4rem 0 0; min-height: 1.2em; }

// Code not found

Increase the number of dice per roll (n) and the histogram quickly snaps into a bell curve, no matter how skewed the single-die distribution is. This is the CLT at work: each column of the histogram is the sum of n independent rolls, and their collective shape converges to a normal distribution as n grows.

The Central Limit Theorem is a proven mathematical result — not an approximation or a conjecture. Here is what it actually states:

Let $X_1, X_2, \ldots, X_n$ be independent and identically distributed (i.i.d.) random variables, each with mean $\mu$ and finite variance $\sigma^2$. Define their standardized sum:

$$Z_n = \frac{X_1 + X_2 + \cdots + X_n - n\mu}{\sigma\sqrt{n}}$$

Then as $n \to \infty$, the distribution of $Z_{n}$ converges in distribution to the standard normal distribution $N(0,1)$. This means that for large n, sums of i.i.d. random variables are approximately normal, regardless of the shape of the original distribution.

Key points:

• Status: proven (Laplace, 1810; rigorous proof by Lyapunov, 1901). This is one of the deepest settled results in probability theory.
• The i.i.d. assumption can be relaxed. Lindeberg (1922) proved a more general version: as long as no single variable dominates the sum, you still get a normal limit. Lyapunov's condition is an easy-to-check sufficient criterion.
• Finite variance is essential. Distributions with infinite variance (like the Cauchy distribution) do NOT obey the CLT. Their sums converge instead to a different family called stable distributions.
• The convergence is in distribution, not almost-surely. Individual sums can still be surprising; only the shape of the histogram is guaranteed to approach the bell curve.
• Speed of convergence: the Berry–Esseen theorem (1941) gives an explicit bound: the error in the normal approximation is at most $O(1/\sqrt{n})$.

The CLT belongs to probability theory's "solved" core — alongside the Law of Large Numbers — and underlies virtually all of classical statistics.

The CLT is arguably the most-used theorem in applied science. Everywhere that "many small independent effects add up", the bell curve appears:

• Polling and elections: a poll of $n$ people is a sum of $n$ independent $\pm 1$ votes. The CLT tells us the margin of error shrinks as $1/\sqrt{n}$ — which is why doubling a poll's accuracy requires quadrupling its size.
• Quality control: a manufactured part's dimension is the sum of many small machining errors. The CLT guarantees these errors follow a normal distribution, making control charts and tolerance analysis tractable.
• Hypothesis testing and confidence intervals: virtually every t-test, z-test, and chi-square test is justified by the CLT. "The sample mean is approximately normal" is the engine under classical inferential statistics.
• Financial models: daily stock returns are sums of many tiny trades. The CLT (and its limits — see fat tails) underlies options pricing and Value-at-Risk calculations.
• Signal processing: electronic noise is the sum of many random electron movements. The CLT explains why thermal noise is Gaussian, justifying the entire field of Gaussian signal processing.
• Machine learning: the gradient in stochastic gradient descent is an average over a mini-batch — the CLT explains why SGD's gradient estimates cluster around their true value as batch size grows.

Understanding the CLT is also the key to recognizing when it fails — heavy-tailed distributions (like those in Bayesian inference) can break the normal approximation, with real consequences for risk models and physics.

The Central Limit Theorem is one of those results that sounds like magic but is rigorously proven: no matter what shape randomness starts with, summing enough of it always produces the bell curve. Laplace saw it in 1810; Lyapunov proved it cleanly in 1901; and ever since, it has been the silent engine under polls, tests, control charts, and models across every quantitative field.

The next time you see a histogram that looks like a bell, ask yourself: what independent effects are being summed? Almost always, the CLT is at work — transforming chaos into the most recognizable curve in all of mathematics.

Curious about what happens when the CLT breaks? Explore Bayesian inference to see how heavy-tailed priors and likelihoods can foil the normal approximation, and what probabilists do instead.