The Math Behind Classifier-Free Guidance

Introduction

Every time you type a prompt and a stunning image appears, a simple arithmetic trick is doing most of the work. It is called classifier-free guidance (CFG), and it was introduced by Jonathan Ho and Tim Salimans in 2022.

A diffusion model learns to reverse noise: given a noisy image, predict the noise that was added. Doing that conditioned on a text prompt should push the result toward your description — but in practice the model drifts. Without extra pressure it hedges, averaging over everything consistent with the words and producing flat, unconvincing samples.

CFG fixes this with a remarkably small change. At each denoising step the model runs twice: once with your prompt and once with an empty prompt (no condition at all). Then it extrapolates — pushes further in the direction the conditional score differs from the unconditional one:

\tilde{\epsilon}_\theta(x_t, c) = \epsilon_\theta(x_t) + w \cdot \bigl(\epsilon_\theta(x_t, c) - \epsilon_\theta(x_t)\bigr)

where $w$ is the guidance scale. At $w = 1$ you get the raw conditional prediction. Push $w$ higher and the output becomes sharper and more faithful to the prompt — at the cost of less variety and eventually over-saturation.

That single number $w$ is the creative control sitting behind the slider in every major image generator today.

Try It

The simulation below runs a toy 1-D diffusion process over a mixture of two Gaussians (think of them as two "concepts" in the model's world). The target concept is the right-hand peak. Drag the guidance scale and press Sample to see where the denoised particles land.

<div class="hint">Two distributions live in this world: the <b>unconditioned</b> mixture (dashed gray, two peaks) and the <b>target</b> (solid blue, right peak). Guidance extrapolates the score away from the unconditioned distribution toward the target. Drag the scale and sample to see the effect.</div>
<div class="controls">
  <label for="guidance">Guidance scale w = <span id="wval">1.0</span></label>
  <input type="range" id="guidance" min="0" max="20" step="0.5" value="1">
</div>
<div class="controls">
  <label for="nsamples">Number of samples = <span id="nsval">40</span></label>
  <input type="range" id="nsamples" min="5" max="80" step="5" value="40">
</div>
<canvas id="canvas" width="480" height="220"></canvas>
<div class="status" id="status"></div>
<div class="btns">
  <button id="btn-sample" type="button">Sample</button>
  <button id="btn-reset" type="button" class="ghost">Reset</button>
</div>

* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; color: #222; margin: 0; }
.hint { font-size: .88rem; color: #444; margin: 0 0 .7rem; line-height: 1.45; }
.controls { display: flex; align-items: center; gap: .6rem; margin-bottom: .4rem; font-size: .9rem; }
.controls label { min-width: 160px; }
input[type=range] { flex: 1; accent-color: #1d3557; }
canvas { display: block; border: 1px solid #cdd9e3; border-radius: 8px;
         background: #f8fafc; width: 100%; max-width: 480px; margin: .5rem 0; }
.status { font-size: .95rem; font-weight: 600; min-height: 1.4em; margin: .3rem 0; }
.status.ok { color: #0a7d33; }
.btns { display: flex; gap: .5rem; flex-wrap: wrap; }
button { font: 600 14px system-ui, sans-serif; padding: .45rem .9rem;
         border: 1px solid #1d3557; background: #1d3557; color: #fff;
         border-radius: 8px; cursor: pointer; }
button.ghost { background: #fff; color: #1d3557; }

const i18nFmt = (s, v) => s.replace(/%(\w+)%/g, (_, k) => String(v[k] ?? ''));
// Classifier-Free Guidance (CFG) demo — 1-D toy diffusion.
// Unconditioned distribution = mixture of two Gaussians at -0.6 and 0.6.
const MU_UNCOND = 0.0;
const MU_TARGET = 1.0;
const SIGMA = 0.28;

// Score function = gradient of log-probability. Used by Langevin dynamics.
function scoreUncond(x) {
  // Mixture-of-Gaussians score (unnormalized; proportional to gradient of log p).
  const w1 = 0.5, w2 = 0.5;
  const mu1 = -0.6, mu2 = 0.6;
  const s1 = Math.exp(-0.5 * Math.pow((x - mu1) / SIGMA, 2));
  const s2 = Math.exp(-0.5 * Math.pow((x - mu2) / SIGMA, 2));
  const denom = w1 * s1 + w2 * s2 + 1e-9;
  return (w1 * s1 * (mu1 - x) + w2 * s2 * (mu2 - x)) / (denom * SIGMA * SIGMA);
}

function scoreCond(x) {
  // Conditional score = gradient of log p(x | c) for the right-peak target.
  return (MU_TARGET - x) / (SIGMA * SIGMA);
}

function cfgScore(x, w) {
  // CFG extrapolation: uncond_score + w * (cond_score - uncond_score).
  return scoreUncond(x) + w * (scoreCond(x) - scoreUncond(x));
}

function sampleLangevin(w, n) {
  // Simple Langevin sampler: x += dt * score(x, w) + noise.
  const dt = 0.05;
  const steps = 60;
  const noise = 0.18;
  const xs = [];
  for (let i = 0; i < n; i++) {
    let x = (Math.random() - 0.5) * 3.0;
    for (let s = 0; s < steps; s++) {
      x += dt * cfgScore(x, w) + noise * Math.sqrt(2 * dt) * (Math.random() * 2 - 1);
    }
    xs.push(x);
  }
  return xs;
}

function gaussian(x, mu, sigma) {
  return Math.exp(-0.5 * Math.pow((x - mu) / sigma, 2)) / (sigma * Math.sqrt(2 * Math.PI));
}

const canvas = document.getElementById('canvas');
const ctx = canvas.getContext('2d');
const W = canvas.width, H = canvas.height;
const XMIN = -2.0, XMAX = 2.0;
const YMAX = 1.6;

function toPixel(x, y) {
  return [
    Math.round(((x - XMIN) / (XMAX - XMIN)) * W),
    Math.round((1 - y / YMAX) * (H - 30) + 5)
  ];
}

function drawDistributions() {
  // Clear canvas before redraw.
  ctx.clearRect(0, 0, W, H);

  // Draw unconditioned distribution as dashed gray curve.
  ctx.beginPath();
  ctx.strokeStyle = '#adb5bd';
  ctx.lineWidth = 1.5;
  ctx.setLineDash([4, 3]);
  for (let px = 0; px < W; px++) {
    const x = XMIN + (px / W) * (XMAX - XMIN);
    const y = 0.5 * gaussian(x, -0.6, SIGMA) + 0.5 * gaussian(x, 0.6, SIGMA);
    const [cx, cy] = toPixel(x, y);
    px === 0 ? ctx.moveTo(cx, cy) : ctx.lineTo(cx, cy);
  }
  ctx.stroke();
  ctx.setLineDash([]);

  // Draw target (conditional) distribution as solid blue curve.
  ctx.beginPath();
  ctx.strokeStyle = '#1d3557';
  ctx.lineWidth = 2;
  for (let px = 0; px < W; px++) {
    const x = XMIN + (px / W) * (XMAX - XMIN);
    const y = gaussian(x, MU_TARGET, SIGMA);
    const [cx, cy] = toPixel(x, y);
    px === 0 ? ctx.moveTo(cx, cy) : ctx.lineTo(cx, cy);
  }
  ctx.stroke();

  // Draw x-axis baseline.
  ctx.beginPath();
  ctx.strokeStyle = '#888';
  ctx.lineWidth = 1;
  const [, axY] = toPixel(0, 0);
  ctx.moveTo(0, axY);
  ctx.lineTo(W, axY);
  ctx.stroke();
}

function drawSamples(xs) {
  // Draw histogram of sampled particles as red bars.
  const bins = new Array(40).fill(0);
  for (const x of xs) {
    const b = Math.floor(((x - XMIN) / (XMAX - XMIN)) * bins.length);
    if (b >= 0 && b < bins.length) bins[b]++;
  }
  const maxB = Math.max(...bins, 1);
  ctx.fillStyle = 'rgba(230, 57, 70, 0.55)';
  for (let i = 0; i < bins.length; i++) {
    if (!bins[i]) continue;
    const xL = XMIN + (i / bins.length) * (XMAX - XMIN);
    const xR = XMIN + ((i + 1) / bins.length) * (XMAX - XMIN);
    const barH = (bins[i] / maxB) * YMAX * 0.85;
    const [px1] = toPixel(xL, 0);
    const [px2, py] = toPixel(xR, barH);
    const [, y0] = toPixel(xL, 0);
    ctx.fillRect(px1, py, px2 - px1 - 1, y0 - py);
  }
}

// Draw legend text in the top-left corner.
function drawLegend(w) {
  ctx.font = '11px system-ui, sans-serif';
  ctx.fillStyle = '#adb5bd';
  ctx.fillText('--- unconditioned (mixture)', 8, 16);
  ctx.fillStyle = '#1d3557';
  ctx.fillText('--- target distribution', 8, 30);
  ctx.fillStyle = 'rgba(230,57,70,0.8)';
  ctx.fillText(i18nFmt('=== samples (w = %w%)', { w: w.toFixed(1) }), 8, 44);
}

let lastSamples = null;

function draw(samples, w) {
  drawDistributions();
  if (samples) drawSamples(samples);
  drawLegend(w);
}

const wSlider = document.getElementById('guidance');
const wVal = document.getElementById('wval');
const nsSlider = document.getElementById('nsamples');
const nsVal = document.getElementById('nsval');
const statusEl = document.getElementById('status');

wSlider.addEventListener('input', () => {
  wVal.textContent = parseFloat(wSlider.value).toFixed(1);
  draw(lastSamples, parseFloat(wSlider.value));
});
nsSlider.addEventListener('input', () => { nsVal.textContent = nsSlider.value; });

document.getElementById('btn-sample').addEventListener('click', () => {
  const w = parseFloat(wSlider.value);
  const n = parseInt(nsSlider.value, 10);
  lastSamples = sampleLangevin(w, n);
  const mean = lastSamples.reduce((a, b) => a + b, 0) / n;
  draw(lastSamples, w);
  const inTarget = lastSamples.filter(x => x > 0.2).length;
  statusEl.className = 'status ok';
  statusEl.textContent = i18nFmt('%n% samples drawn at w = %w% — %pct%% landed near the target.', { n: n, pct: Math.round(inTarget / n * 100), w: w.toFixed(1) });
});

document.getElementById('btn-reset').addEventListener('click', () => {
  lastSamples = null;
  wSlider.value = 1;
  wVal.textContent = '1.0';
  nsSlider.value = 40;
  nsVal.textContent = '40';
  statusEl.className = 'status';
  statusEl.textContent = '';
  draw(null, 1.0);
});

draw(null, 1.0);

<div class="hint">Two distributions live in this world: the <b>unconditioned</b> mixture (dashed gray, two peaks) and the <b>target</b> (solid blue, right peak). Guidance extrapolates the score away from the unconditioned distribution toward the target. Drag the scale and sample to see the effect.</div>
<div class="controls">
  <label for="guidance">Guidance scale w = <span id="wval">1.0</span></label>
  <input type="range" id="guidance" min="0" max="20" step="0.5" value="1">
</div>
<div class="controls">
  <label for="nsamples">Number of samples = <span id="nsval">40</span></label>
  <input type="range" id="nsamples" min="5" max="80" step="5" value="40">
</div>
<canvas id="canvas" width="480" height="220"></canvas>
<div class="status" id="status"></div>
<div class="btns">
  <button id="btn-sample" type="button">Sample</button>
  <button id="btn-reset" type="button" class="ghost">Reset</button>
</div>

* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; color: #222; margin: 0; }
.hint { font-size: .88rem; color: #444; margin: 0 0 .7rem; line-height: 1.45; }
.controls { display: flex; align-items: center; gap: .6rem; margin-bottom: .4rem; font-size: .9rem; }
.controls label { min-width: 160px; }
input[type=range] { flex: 1; accent-color: #1d3557; }
canvas { display: block; border: 1px solid #cdd9e3; border-radius: 8px;
         background: #f8fafc; width: 100%; max-width: 480px; margin: .5rem 0; }
.status { font-size: .95rem; font-weight: 600; min-height: 1.4em; margin: .3rem 0; }
.status.ok { color: #0a7d33; }
.btns { display: flex; gap: .5rem; flex-wrap: wrap; }
button { font: 600 14px system-ui, sans-serif; padding: .45rem .9rem;
         border: 1px solid #1d3557; background: #1d3557; color: #fff;
         border-radius: 8px; cursor: pointer; }
button.ghost { background: #fff; color: #1d3557; }

const i18nFmt = (s, v) => s.replace(/%(\w+)%/g, (_, k) => String(v[k] ?? ''));
// Classifier-Free Guidance (CFG) demo — 1-D toy diffusion.
// Unconditioned distribution = mixture of two Gaussians at -0.6 and 0.6.
const MU_UNCOND = 0.0;
const MU_TARGET = 1.0;
const SIGMA = 0.28;

// Score function = gradient of log-probability. Used by Langevin dynamics.
function scoreUncond(x) {
  // Mixture-of-Gaussians score (unnormalized; proportional to gradient of log p).
  const w1 = 0.5, w2 = 0.5;
  const mu1 = -0.6, mu2 = 0.6;
  const s1 = Math.exp(-0.5 * Math.pow((x - mu1) / SIGMA, 2));
  const s2 = Math.exp(-0.5 * Math.pow((x - mu2) / SIGMA, 2));
  const denom = w1 * s1 + w2 * s2 + 1e-9;
  return (w1 * s1 * (mu1 - x) + w2 * s2 * (mu2 - x)) / (denom * SIGMA * SIGMA);
}

function scoreCond(x) {
  // Conditional score = gradient of log p(x | c) for the right-peak target.
  return (MU_TARGET - x) / (SIGMA * SIGMA);
}

function cfgScore(x, w) {
  // CFG extrapolation: uncond_score + w * (cond_score - uncond_score).
  return scoreUncond(x) + w * (scoreCond(x) - scoreUncond(x));
}

function sampleLangevin(w, n) {
  // Simple Langevin sampler: x += dt * score(x, w) + noise.
  const dt = 0.05;
  const steps = 60;
  const noise = 0.18;
  const xs = [];
  for (let i = 0; i < n; i++) {
    let x = (Math.random() - 0.5) * 3.0;
    for (let s = 0; s < steps; s++) {
      x += dt * cfgScore(x, w) + noise * Math.sqrt(2 * dt) * (Math.random() * 2 - 1);
    }
    xs.push(x);
  }
  return xs;
}

function gaussian(x, mu, sigma) {
  return Math.exp(-0.5 * Math.pow((x - mu) / sigma, 2)) / (sigma * Math.sqrt(2 * Math.PI));
}

const canvas = document.getElementById('canvas');
const ctx = canvas.getContext('2d');
const W = canvas.width, H = canvas.height;
const XMIN = -2.0, XMAX = 2.0;
const YMAX = 1.6;

function toPixel(x, y) {
  return [
    Math.round(((x - XMIN) / (XMAX - XMIN)) * W),
    Math.round((1 - y / YMAX) * (H - 30) + 5)
  ];
}

function drawDistributions() {
  // Clear canvas before redraw.
  ctx.clearRect(0, 0, W, H);

  // Draw unconditioned distribution as dashed gray curve.
  ctx.beginPath();
  ctx.strokeStyle = '#adb5bd';
  ctx.lineWidth = 1.5;
  ctx.setLineDash([4, 3]);
  for (let px = 0; px < W; px++) {
    const x = XMIN + (px / W) * (XMAX - XMIN);
    const y = 0.5 * gaussian(x, -0.6, SIGMA) + 0.5 * gaussian(x, 0.6, SIGMA);
    const [cx, cy] = toPixel(x, y);
    px === 0 ? ctx.moveTo(cx, cy) : ctx.lineTo(cx, cy);
  }
  ctx.stroke();
  ctx.setLineDash([]);

  // Draw target (conditional) distribution as solid blue curve.
  ctx.beginPath();
  ctx.strokeStyle = '#1d3557';
  ctx.lineWidth = 2;
  for (let px = 0; px < W; px++) {
    const x = XMIN + (px / W) * (XMAX - XMIN);
    const y = gaussian(x, MU_TARGET, SIGMA);
    const [cx, cy] = toPixel(x, y);
    px === 0 ? ctx.moveTo(cx, cy) : ctx.lineTo(cx, cy);
  }
  ctx.stroke();

  // Draw x-axis baseline.
  ctx.beginPath();
  ctx.strokeStyle = '#888';
  ctx.lineWidth = 1;
  const [, axY] = toPixel(0, 0);
  ctx.moveTo(0, axY);
  ctx.lineTo(W, axY);
  ctx.stroke();
}

function drawSamples(xs) {
  // Draw histogram of sampled particles as red bars.
  const bins = new Array(40).fill(0);
  for (const x of xs) {
    const b = Math.floor(((x - XMIN) / (XMAX - XMIN)) * bins.length);
    if (b >= 0 && b < bins.length) bins[b]++;
  }
  const maxB = Math.max(...bins, 1);
  ctx.fillStyle = 'rgba(230, 57, 70, 0.55)';
  for (let i = 0; i < bins.length; i++) {
    if (!bins[i]) continue;
    const xL = XMIN + (i / bins.length) * (XMAX - XMIN);
    const xR = XMIN + ((i + 1) / bins.length) * (XMAX - XMIN);
    const barH = (bins[i] / maxB) * YMAX * 0.85;
    const [px1] = toPixel(xL, 0);
    const [px2, py] = toPixel(xR, barH);
    const [, y0] = toPixel(xL, 0);
    ctx.fillRect(px1, py, px2 - px1 - 1, y0 - py);
  }
}

// Draw legend text in the top-left corner.
function drawLegend(w) {
  ctx.font = '11px system-ui, sans-serif';
  ctx.fillStyle = '#adb5bd';
  ctx.fillText('--- unconditioned (mixture)', 8, 16);
  ctx.fillStyle = '#1d3557';
  ctx.fillText('--- target distribution', 8, 30);
  ctx.fillStyle = 'rgba(230,57,70,0.8)';
  ctx.fillText(i18nFmt('=== samples (w = %w%)', { w: w.toFixed(1) }), 8, 44);
}

let lastSamples = null;

function draw(samples, w) {
  drawDistributions();
  if (samples) drawSamples(samples);
  drawLegend(w);
}

const wSlider = document.getElementById('guidance');
const wVal = document.getElementById('wval');
const nsSlider = document.getElementById('nsamples');
const nsVal = document.getElementById('nsval');
const statusEl = document.getElementById('status');

wSlider.addEventListener('input', () => {
  wVal.textContent = parseFloat(wSlider.value).toFixed(1);
  draw(lastSamples, parseFloat(wSlider.value));
});
nsSlider.addEventListener('input', () => { nsVal.textContent = nsSlider.value; });

document.getElementById('btn-sample').addEventListener('click', () => {
  const w = parseFloat(wSlider.value);
  const n = parseInt(nsSlider.value, 10);
  lastSamples = sampleLangevin(w, n);
  const mean = lastSamples.reduce((a, b) => a + b, 0) / n;
  draw(lastSamples, w);
  const inTarget = lastSamples.filter(x => x > 0.2).length;
  statusEl.className = 'status ok';
  statusEl.textContent = i18nFmt('%n% samples drawn at w = %w% — %pct%% landed near the target.', { n: n, pct: Math.round(inTarget / n * 100), w: w.toFixed(1) });
});

document.getElementById('btn-reset').addEventListener('click', () => {
  lastSamples = null;
  wSlider.value = 1;
  wVal.textContent = '1.0';
  nsSlider.value = 40;
  nsVal.textContent = '40';
  statusEl.className = 'status';
  statusEl.textContent = '';
  draw(null, 1.0);
});

draw(null, 1.0);

Result

Console

At $w = 0$ (no guidance) samples scatter across both peaks — the model is unguided. As you raise $w$ the particles migrate to the target peak and cluster tightly. Push $w$ too high and they overshoot: the distribution collapses to a single spike and diversity vanishes. This is exactly the quality-diversity trade-off that practitioners tune when generating images.

The Math Behind It

The formula is short, but the reasoning behind it runs deep.

Score functions and Bayes' theorem. A diffusion model implicitly estimates the score $\nabla_{x_t} \log p(x_t)$ — the gradient pointing toward higher-probability images. A conditional model estimates $\nabla_{x_t} \log p(x_t \mid c)$ . By Bayes' theorem:

\nabla_{x_t} \log p(x_t \mid c) = \nabla_{x_t} \log p(x_t) + \nabla_{x_t} \log p(c \mid x_t)

The second term is exactly what a classifier trained on noisy images would provide — pushing the score toward images that a classifier would label as class $c$ . The original classifier guidance (Dhariwal & Nichol, 2021) literally trained such a classifier and added its gradient. The problem: you need a separate noise-robust classifier for every concept.

The classifier-free trick. Ho & Salimans realized you can implicitly compute $\nabla_{x_t} \log p(c \mid x_t)$ without any classifier: just subtract the unconditional score from the conditional score. Scale that difference by $w$ and add it back:

\tilde{\epsilon} = \epsilon_\theta(x_t) + w \cdot \bigl(\epsilon_\theta(x_t, c) - \epsilon_\theta(x_t)\bigr)

This is equivalent to classifier guidance with a "virtual" classifier whose gradient is approximated by the model's own conditional-vs-unconditional gap.

The cost. Every step needs two forward passes instead of one — roughly doubling inference time. Practitioners commonly set $w$ between 7 and 15 for photorealistic models; going much higher produces artifacts (the model enters regions of the distribution it was never trained on).

Why it works so well. The guidance is applied at every denoising step, so the small nudge compounds: each step reinforces the previous ones and the trajectory bends steadily toward the target region. It is the same principle as gradient ascent — repeated small pushes in the right direction.

Where It Matters

CFG is not a research curiosity — it is the workhorse behind nearly every modern generative system:

Text-to-image: Stable Diffusion, DALL-E 3, Imagen, Flux — all use CFG or a close variant. The guidance scale is usually the first parameter a user is told to tweak.
Text-to-video: Sora, RunwayML, and similar models extend the same trick to the temporal dimension. A high guidance scale keeps shots coherent to the prompt across frames.
Audio generation: Diffusion-based audio models (MusicLM variants, AudioLDM) apply CFG on mel-spectrograms to steer music or speech toward a text description.
Protein and molecule design: Protein folding researchers use guidance to bias diffusion trajectories toward structures with desired properties — binding sites, thermostability — without retraining the backbone model.
Conditional scientific simulation: climate and fluid models trained as diffusion processes can be guided toward physically meaningful states using the same extrapolation.

The key insight — steer by extrapolating away from the unconstrained distribution — is so clean and general that it has been adopted far beyond image synthesis. Any generative model that can run both conditioned and unconditioned can gain controllability with CFG.

Conclusion

Classifier-free guidance is a beautiful example of a small idea with enormous reach. Two forward passes, a subtraction, a scalar — and a diffusion model suddenly listens to your prompt.

The deeper lesson is geometric: the unconditioned model knows the shape of the whole distribution, and the conditioned model knows the direction of your target. Extrapolating between them is just a way of saying "go further in the direction the condition points." Every image generator with a "CFG scale" slider is running that arithmetic thousands of times per second.

The trade-off it encodes — fidelity vs. diversity — is not a bug but a feature of how probability distributions work. Push $w$ to infinity and you get the most likely single sample, perfectly faithful but utterly brittle. Stay near $w = 1$ and you roam the full distribution, creative but unfocused. The art is in choosing $w$ wisely, which is why it remains one of the most-discussed numbers in neural network training and generative AI.

Comments

Loading comments...

https://www.kipuhub.com/en/article/diffusion-guidance-cfg/Content licensed under CC BY-NC 4.0.

Cookies & privacy

Classifier-Free Guidance

One number that trades diversity for fidelity