Introduction

Every time you ask a voice assistant a question, the first thing it does is throw away most of your audio. Not carelessly — very carefully. It distills each tiny slice of sound into about thirteen numbers that capture everything a human ear cares about and nothing it does not.

Those thirteen numbers are a Mel-Frequency Cepstral Coefficient (MFCC) vector. They have been the dominant speech feature since Davis and Mermelstein introduced them in 1980, and even modern deep-learning systems still start there or build on the same perceptual principles.

The key insight is that the ear is not a microphone. Pitch differences sound smaller and smaller the higher they go — an octave at low frequencies feels much bigger than an octave at high frequencies. Any feature that ignores this will waste most of its budget encoding detail the ear cannot perceive. MFCCs do not make that mistake: they warp the frequency axis to match human perception before doing any analysis, so every coefficient counts.

Try It

Select a vowel below to see its idealized power spectrum. The demo walks through the MFCC pipeline one step at a time: warp the frequency axis onto the mel scale, apply triangular filterbank bins, take the log, and finally a Discrete Cosine Transform to decorrelate the channels into the final coefficients.

<!-- MFCC pipeline demo: power spectrum -> mel filterbank -> log -> DCT -> 13 coefficients -->
<div class="controls">
  <label for="vowel-sel">Vowel:</label>
  <select id="vowel-sel">
    <option value="a">/a/ (father)</option>
    <option value="e">/e/ (bed)</option>
    <option value="i">/i/ (see)</option>
    <option value="o">/o/ (go)</option>
    <option value="u">/u/ (too)</option>
  </select>
  <button id="btn-reset" type="button" class="ghost">Reset</button>
</div>
<div class="steps">
  <button id="btn-prev" type="button" class="nav" disabled>&#8592;</button>
  <span id="step-label" class="step-label"></span>
  <button id="btn-next" type="button" class="nav">&#8594;</button>
</div>
<div class="desc-box" id="step-desc"></div>
<canvas id="chart" width="560" height="200"></canvas>
<div class="mfcc-row" id="mfcc-row" style="display:none">
  <div class="mfcc-label">MFCC vector (13 coefficients):</div>
  <div id="mfcc-bars" class="mfcc-bars"></div>
</div>

/* Demo layout: controls on top, step navigator, description, canvas chart, MFCC bars */
* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; color: #222; margin: 0; background: #fff; }
.controls { display: flex; align-items: center; gap: .6rem; margin-bottom: .6rem; flex-wrap: wrap; }
label { font-size: .9rem; font-weight: 600; }
select { font-size: .9rem; padding: .3rem .5rem; border: 1px solid #bbb; border-radius: 6px; }
button { font: 600 13px system-ui; padding: .35rem .75rem; border: 1px solid #1d3557;
         background: #1d3557; color: #fff; border-radius: 6px; cursor: pointer; }
button.ghost { background: #fff; color: #1d3557; }
button:disabled { opacity: .35; cursor: default; }
.steps { display: flex; align-items: center; gap: .5rem; margin-bottom: .4rem; }
.step-label { font-size: .92rem; font-weight: 700; flex: 1; text-align: center; }
.nav { padding: .3rem .65rem; font-size: 1rem; min-width: 36px; }
.desc-box { font-size: .84rem; color: #444; line-height: 1.5; margin-bottom: .5rem;
            min-height: 2.6rem; padding: .35rem .6rem; background: #f4f7fa;
            border-left: 3px solid #1d3557; border-radius: 4px; }
canvas { width: 100%; max-width: 560px; display: block; border: 1px solid #e0e5ea; border-radius: 8px; background: #fafbfc; }
/* Compact MFCC coefficient bar strip shown at step 4 */
.mfcc-row { margin-top: .6rem; }
.mfcc-label { font-size: .8rem; font-weight: 700; color: #555; margin-bottom: .25rem; }
.mfcc-bars { display: flex; gap: 3px; align-items: flex-end; height: 54px; }
.mfcc-bar { flex: 1; border-radius: 3px 3px 0 0; min-height: 4px; transition: height .35s; }

/* MFCC pipeline: power spectrum -> mel filterbank -> log energy -> DCT coefficients */
const i18nFmt = (s, v) => s.replace(/%(\w+)%/g, (_, k) => String(v[k] ?? ''));

/* Approximate formant frequencies (F1, F2, F3) in Hz for five English vowels */
const VOWELS = {
  a: { f1: 730, f2: 1090, f3: 2440 },
  e: { f1: 530, f2: 1840, f3: 2480 },
  i: { f1: 270, f2: 2290, f3: 3010 },
  o: { f1: 570, f2:  840, f3: 2410 },
  u: { f1: 300, f2:  870, f3: 2240 }
};

/* Standard mel-scale conversion: mel = 2595 * log10(1 + f/700) */
const hz2mel = f => 2595 * Math.log10(1 + f / 700);
const mel2hz = m => 700 * (Math.pow(10, m / 2595) - 1);

const N_FFT = 128;   // number of simulated FFT power spectrum bins
const N_MELS = 20;   // number of triangular mel filterbank channels
const N_MFCC = 13;   // number of MFCC coefficients to keep
const F_MAX = 8000;  // maximum frequency in Hz

/* Build an idealized power spectrum for the given vowel using Gaussian formant peaks */
function makePowerSpectrum(vowel) {
  const { f1, f2, f3 } = VOWELS[vowel];
  const spec = new Float32Array(N_FFT);
  const fStep = F_MAX / N_FFT;
  for (let k = 0; k < N_FFT; k++) {
    const f = (k + 0.5) * fStep;
    // three formant peaks modeled as Gaussians — three formant peaks modeled as Gaussians
    const g1 = Math.exp(-Math.pow((f - f1) / 150, 2));
    const g2 = Math.exp(-Math.pow((f - f2) / 250, 2)) * 0.7;
    const g3 = Math.exp(-Math.pow((f - f3) / 350, 2)) * 0.4;
    spec[k] = Math.max(0.01, g1 + g2 + g3);
  }
  return spec;
}

/* Build a bank of N_MELS triangular filters equally spaced on the mel scale */
function buildFilterbank() {
  const melMin = hz2mel(0);
  const melMax = hz2mel(F_MAX);
  const melPoints = [];
  for (let m = 0; m <= N_MELS + 1; m++)
    melPoints.push(mel2hz(melMin + (m / (N_MELS + 1)) * (melMax - melMin)));

  const fb = [];
  const fStep = F_MAX / N_FFT;
  for (let m = 1; m <= N_MELS; m++) {
    const row = new Float32Array(N_FFT);
    for (let k = 0; k < N_FFT; k++) {
      const f = (k + 0.5) * fStep;
      const lo = melPoints[m - 1], cen = melPoints[m], hi = melPoints[m + 1];
      if (f >= lo && f <= cen) row[k] = (f - lo) / (cen - lo);
      else if (f > cen && f <= hi) row[k] = (hi - f) / (hi - cen);
    }
    fb.push(row);
  }
  return fb;
}

/* Multiply the power spectrum by each filter row and sum (dot product per channel) */
function applyFilterbank(spec, fb) {
  return fb.map(row => {
    let s = 0;
    for (let k = 0; k < N_FFT; k++) s += row[k] * spec[k];
    return s;
  });
}

/* Type-II DCT: decorrelates the log filterbank energies into cepstral coefficients */
function dct(x, C) {
  const out = [];
  for (let n = 0; n < C; n++) {
    let s = 0;
    for (let k = 0; k < x.length; k++)
      s += x[k] * Math.cos(Math.PI * n * (k + 0.5) / x.length);
    out.push(s);
  }
  return out;
}

const canvas = document.getElementById('chart');
const ctx = canvas.getContext('2d');
const FB = buildFilterbank();

let currentVowel = 'a';
let currentStep = 0; // 0=power spectrum, 1=mel filterbank, 2=log energy, 3=MFCC coefficients

/* Keys matching step index to i18n description keys */
const STEP_KEYS = ['step_power', 'step_mel', 'step_log', 'step_mfcc'];

function getStepDesc(key) {
  return {
    step_power: 'The raw power spectrum <b>|X(f)|²</b> shows energy at each frequency bin. Vowels have distinct peaks called <b>formants</b> (F1, F2, F3) that define their sound. The x-axis spans 0–8 kHz; high-frequency detail is perceptually less important.',
    step_mel:   'A bank of <b>20 triangular filters</b> is applied, spaced evenly on the <b>mel scale</b>. Bins are narrow at low frequencies (where the ear is most sensitive) and wide at high frequencies. Each bar is the total energy captured by one filter.',
    step_log:   'The <b>logarithm</b> of each filterbank energy compresses the dynamic range to match how loudness is perceived. It also turns the vocal-tract convolution into a sum — separating the speaker’s voice from the message content.',
    step_mfcc:  'A <b>Discrete Cosine Transform</b> decorrelates the 20 log energies into 13 MFCC coefficients. Adjacent mel channels are correlated; the DCT removes that redundancy. The first few coefficients capture the broad spectral envelope; later ones encode finer detail.'
  }[key] || '';
}

function computeSteps(vowel) {
  const spec = makePowerSpectrum(vowel);
  const melEnergy = applyFilterbank(spec, FB);
  const logEnergy = melEnergy.map(v => Math.log(v + 1e-8));
  const mfcc = dct(logEnergy, N_MFCC);
  return { spec, melEnergy, logEnergy, mfcc };
}

/* Generic bar chart renderer: values array, fill color, axis labels */
function drawBars(values, color, xLabel, yLabel) {
  const W = canvas.width, H = canvas.height;
  const pad = { left: 42, right: 10, top: 18, bottom: 32 };
  const plotW = W - pad.left - pad.right;
  const plotH = H - pad.top - pad.bottom;
  ctx.clearRect(0, 0, W, H);

  const minV = Math.min(...values);
  const maxV = Math.max(...values);
  const range = maxV - minV || 1;

  ctx.fillStyle = '#e8eef3';
  ctx.fillRect(pad.left, pad.top, plotW, plotH);

  const barW = plotW / values.length;
  values.forEach((v, i) => {
    const normH = ((v - minV) / range) * plotH;
    ctx.fillStyle = color;
    ctx.fillRect(pad.left + i * barW + 1, pad.top + plotH - normH, barW - 2, normH);
  });

  // Draw x-axis and y-axis labels
  ctx.fillStyle = '#555';
  ctx.font = '11px system-ui';
  ctx.textAlign = 'center';
  ctx.fillText(xLabel, pad.left + plotW / 2, H - 4);
  ctx.save();
  ctx.translate(12, pad.top + plotH / 2);
  ctx.rotate(-Math.PI / 2);
  ctx.fillText(yLabel, 0, 0);
  ctx.restore();
}

function update() {
  const data = computeSteps(currentVowel);
  const stepKey = STEP_KEYS[currentStep];

  document.getElementById('step-label').textContent =
    { step_power: 'Step 1 — Power Spectrum', step_mel: 'Step 2 — Mel Filterbank',
      step_log: 'Step 3 — Log Energy', step_mfcc: 'Step 4 — MFCC Coefficients' }[stepKey] || '';

  document.getElementById('step-desc').innerHTML = getStepDesc(stepKey);

  const mfccRow = document.getElementById('mfcc-row');
  mfccRow.style.display = currentStep === 3 ? 'block' : 'none';

  if (currentStep === 0) {
    drawBars(Array.from(data.spec), '#4a7faa', 'Frequency (Hz)', 'Power');
  } else if (currentStep === 1) {
    drawBars(data.melEnergy, '#2a9d5c', 'Mel filterbank channel', 'Energy');
  } else if (currentStep === 2) {
    drawBars(data.logEnergy, '#e07a1a', 'Mel filterbank channel', 'Log energy');
  } else {
    drawBars(data.mfcc, '#6a4cbf', 'MFCC coefficient index', 'Amplitude');
    renderMfccBars(data.mfcc);
  }

  document.getElementById('btn-prev').disabled = currentStep === 0;
  document.getElementById('btn-next').disabled = currentStep === STEP_KEYS.length - 1;
}

/* Render the compact colored MFCC coefficient strip below the chart */
function renderMfccBars(mfcc) {
  const container = document.getElementById('mfcc-bars');
  container.innerHTML = '';
  const maxAbs = Math.max(...mfcc.map(Math.abs), 1);
  const COLORS = ['#1d3557','#457b9d','#2a9d5c','#e07a1a','#6a4cbf',
                  '#c62828','#00838f','#558b2f','#f57f17','#4527a0',
                  '#00695c','#283593','#6d4c41'];
  mfcc.forEach((v, i) => {
    const bar = document.createElement('div');
    bar.className = 'mfcc-bar';
    bar.title = i18nFmt('c%n%: %v%', { n: i + 1, v: v.toFixed(2) });
    const pct = Math.abs(v) / maxAbs;
    bar.style.height = (pct * 48 + 4) + 'px';
    bar.style.background = COLORS[i % COLORS.length];
    container.appendChild(bar);
  });
}

document.getElementById('vowel-sel').addEventListener('change', e => {
  currentVowel = e.target.value;
  update();
});
document.getElementById('btn-prev').addEventListener('click', () => {
  if (currentStep > 0) { currentStep--; update(); }
});
document.getElementById('btn-next').addEventListener('click', () => {
  if (currentStep < STEP_KEYS.length - 1) { currentStep++; update(); }
});
document.getElementById('btn-reset').addEventListener('click', () => {
  currentStep = 0;
  document.getElementById('vowel-sel').value = 'a';
  currentVowel = 'a';
  update();
});

update();
Result
Console

Notice how the filterbank bins get wider at high frequencies — that is the mel scale at work, spending more resolution where the ear is sensitive. The final bar chart is the MFCC vector: a compact fingerprint that carries far more speech information per number than raw FFT bins would.

The Real Complexity

MFCCs are a solved, deterministic algorithm — not an open problem — but each step encodes a non-obvious insight.

Step 1 — Frame the signal. Speech is not stationary, so the waveform is split into overlapping windows of about 25 ms (long enough to capture a full pitch cycle, short enough to treat as locally stable).

Step 2 — Compute the power spectrum. A short-time Fourier transform gives the energy at each frequency bin, producing a spectrum ∣X(f)∣2|X(f)|^2.

Step 3 — Warp to the mel scale. The mel scale approximates the ear's frequency resolution. The conversion from Hz to mel is:

m=2595⋅log⁡10 ⁣(1+f700)m = 2595 \cdot \log_{10}\!\left(1 + \frac{f}{700}\right)

Below about 1 kHz the mel scale is nearly linear; above it, it compresses. A set of KK triangular filters (typically K=26K = 26) are spaced evenly on this warped axis and summed over the spectrum.

Step 4 — Take the log. Humans perceive loudness logarithmically. The log also turns the convolution of the vocal-tract filter with the glottal source into an addition, which the next step can then separate.

Step 5 — Discrete Cosine Transform. Applying a DCT to the KK log filterbank energies decorrelates them (adjacent mel channels are correlated, which wastes model capacity) and produces the cepstral coefficients. Keeping only the first CC coefficients (typically C=13C = 13) discards fine spectral detail that carries little speech information.

The result is a compact vector that is robust to small pitch shifts, recording conditions, and speaker identity — exactly what a speech recognizer needs. Compare this with dimensionality reduction: MFCCs are a hand-crafted, domain-specific reduction designed before machine learning could learn features end-to-end.

Where It Matters

MFCCs turned speech recognition from a research curiosity into a practical technology, and their influence has only grown:

  • Automatic speech recognition (ASR): from the HMM-GMM systems of the 1990s and 2000s to the first deep-learning ASR models, MFCCs were the universal front end. Systems like Dragon NaturallySpeaking ran entirely on them.
  • Speaker identification and verification: a speaker's vocal-tract shape shows up in the MFCC envelope. Systems verify identities by comparing MFCC distributions across utterances.
  • Music information retrieval: MFCCs describe timbre rather than pitch, making them useful for instrument classification, genre detection, and song similarity.
  • Keyword spotting: tiny on-device models for wake-word detection ("Hey Siri", "OK Google") often compute MFCCs directly in firmware on a microcontroller.
  • Modern neural audio: even when networks learn features end-to-end, they usually operate on mel-spectrograms — the step just before the DCT — because the perceptual warping is still beneficial.

The pipeline introduced in 1980 remains the conceptual backbone of audio machine learning. Understanding MFCCs is also a gateway to dimensionality reduction: both fields ask "how do you throw away the most information while keeping the most meaning?"

Conclusion

Mel-frequency cepstral coefficients are a beautiful example of domain knowledge turned into engineering. By asking "what does the ear actually care about?" before touching the data, Davis and Mermelstein compressed the entire information content of a speech sound into thirteen numbers that are genuinely more useful than thousands of raw FFT bins.

That principle — warp your representation to match the structure of the problem, then discard what is irrelevant — did not become obsolete when deep learning arrived. Modern speech systems still start from mel-spectrograms, and the best neural audio models bake in the same perceptual priors. The next time a voice assistant understands you perfectly in a noisy room, a 44-year-old formula is quietly doing its part.

Share this article

Pick a channel — or use your device's native share sheet.

Comments

Loading comments...

https://www.kipuhub.com/en/article/mfcc/Content licensed under CC BY-NC 4.0.