The Algorithm That Learns to Erase Noise

Introduction

Every time you put on noise-cancelling headphones, a small algorithm is running dozens of thousands of times per second with one goal: subtract the hum before it reaches your ears. It does not have a recording of the noise. It does not know the signal it is protecting. It learns on the fly.

The key idea is correlation. Engine rumble, electrical hum, and ventilation roar share a crucial property: a microphone placed near the noise source picks up almost the same interference that will contaminate the signal microphone a moment later. If you can model how the noise travels from one point to the other — the so-called noise path — you can generate a near-perfect copy and subtract it.

The LMS (Least Mean Squares) algorithm, introduced by Bernard Widrow and Marcian Hoff in 1960, does exactly this. It maintains a short filter whose weights it nudges a tiny step in the direction that reduces the error between the combined signal and zero. Step by step, iteration by iteration, the filter converges — and the hum disappears. This is adaptive filtering: a solved problem with a clean algorithm that works in real time.

Try It

Below, a clean signal (a speech-like sine burst) is buried under a steady hum (50 Hz). A reference microphone provides a clean copy of the hum. The LMS filter learns the noise path and subtracts it in real time. Press Run to start, adjust the step size $\mu$ to see how fast (or how unstably) the filter converges, and press Reset to start over.

<!-- LMS adaptive noise cancellation demo: a filter learns the hum path and subtracts it. -->
<div class="controls">
  <label>Step size μ: <input id="mu" type="range" min="0.001" max="0.05" step="0.001" value="0.01"></label>
  <span id="mu-val">0.010</span>
  <button id="run-btn" type="button">Run</button>
  <button id="reset-btn" type="button" class="ghost">Reset</button>
</div>
<canvas id="cnv" width="560" height="280" aria-label="Signal waveforms — noisy input, clean signal, and LMS filter output"></canvas>
<div class="legend">
  <span class="leg-noisy">Noisy input</span>
  <span class="leg-clean">Clean signal</span>
  <span class="leg-output">LMS output</span>
</div>
<div class="status" id="status">Press Run to start the LMS filter.</div>

* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; margin: 0; color: #222; }
.controls { display: flex; align-items: center; gap: .7rem; flex-wrap: wrap; margin-bottom: .6rem; }
label { font-size: .9rem; display: flex; align-items: center; gap: .4rem; }
input[type=range] { width: 120px; }
#mu-val { font-size: .9rem; font-weight: 700; width: 3.5ch; }
button { font: 600 14px system-ui, sans-serif; padding: .45rem .9rem;
         border: 1px solid #1d3557; background: #1d3557; color: #fff;
         border-radius: 8px; cursor: pointer; }
button.ghost { background: #fff; color: #1d3557; }
canvas { display: block; border: 1px solid #cdd9e3; border-radius: 8px;
         width: 100%; max-width: 560px; background: #f8fafc; }
.legend { display: flex; gap: 1rem; font-size: .82rem; margin-top: .4rem; flex-wrap: wrap; }
.legend span { display: flex; align-items: center; gap: .3rem; }
.leg-noisy::before { content: ""; display: inline-block; width: 20px; height: 3px; background: #e63946; border-radius: 2px; }
.leg-clean::before  { content: ""; display: inline-block; width: 20px; height: 3px; background: #457b9d; border-radius: 2px; }
.leg-output::before { content: ""; display: inline-block; width: 20px; height: 3px; background: #2a9d8f; border-radius: 2px; }
.status { font-size: .95rem; font-weight: 600; margin-top: .5rem; min-height: 1.4em; color: #444; }
.status.running { color: #0a7d33; }
.status.done    { color: #0a7d33; }

const i18nFmt = (s, v) => s.replace(/%(\w+)%/g, (_, k) => String(v[k] ?? ''));

// Simulation parameters
const SAMPLE_RATE = 200;   // samples per second (conceptual — actual playback is visual)
const N_TOTAL    = 600;    // total samples to process
const L          = 8;      // FIR filter taps (weights): longer = better at complex paths, slower
const HUM_FREQ   = 0.25;   // normalized hum frequency (50 Hz / 200 Hz sample rate = 0.25)
const SIG_FREQ   = 0.07;   // normalized signal frequency (speech-like low freq)
const SIG_START  = 100;    // sample where the signal burst starts
const SIG_END    = 500;    // sample where the signal burst ends
const HUM_AMP    = 1.0;
const SIG_AMP    = 0.6;

// Generate the three signal arrays: reference hum, clean signal, noisy mix.
function genSignals() {
  const ref    = new Float64Array(N_TOTAL);
  const clean  = new Float64Array(N_TOTAL);
  const noisy  = new Float64Array(N_TOTAL);
  const delay  = 3;   // the hum picked up at the signal mic is delayed + phase-shifted vs. the reference
  for (let n = 0; n < N_TOTAL; n++) {
    ref[n]   = HUM_AMP * Math.sin(2 * Math.PI * HUM_FREQ * n);
    const sig = (n >= SIG_START && n < SIG_END)
                ? SIG_AMP * Math.sin(2 * Math.PI * SIG_FREQ * n) : 0;
    clean[n] = sig;
    const hum = HUM_AMP * Math.sin(2 * Math.PI * HUM_FREQ * (n - delay) + 0.3);
    noisy[n] = sig + hum;
  }
  return { ref, clean, noisy };
}

// Run the LMS adaptive filter over all samples and return the output (error) signal.
function runLMS(ref, noisy, mu) {
  const w   = new Float64Array(L);
  const out = new Float64Array(N_TOTAL);
  for (let n = 0; n < N_TOTAL; n++) {
    // Build the reference input vector x(n) = [ref(n), ref(n-1), ..., ref(n-L+1)]
    const x = new Float64Array(L);
    for (let k = 0; k < L; k++) x[k] = n - k >= 0 ? ref[n - k] : 0;
    // Compute filter output y(n) = w^T * x(n)
    let y = 0;
    for (let k = 0; k < L; k++) y += w[k] * x[k];
    const e = noisy[n] - y;  // error = desired - filter output; this is also the cleaned signal
    out[n] = e;
    // LMS weight update: w(n+1) = w(n) + 2*mu*e(n)*x(n)
    for (let k = 0; k < L; k++) w[k] += 2 * mu * e * x[k];
  }
  return out;
}

const canvas  = document.getElementById('cnv');
const ctx     = canvas.getContext('2d');
const muInput = document.getElementById('mu');
const muVal   = document.getElementById('mu-val');
const runBtn  = document.getElementById('run-btn');
const resetBtn= document.getElementById('reset-btn');
const statusEl= document.getElementById('status');

muInput.addEventListener('input', () => { muVal.textContent = parseFloat(muInput.value).toFixed(3); });

let rafId = null;
let step  = 0;
let signals = null;
let output  = null;
const W = 560, H = 280;
const MID = H / 2;
const SCALE = 80; // amplitude to pixels

function drawFrame() {
  ctx.clearRect(0, 0, W, H);
  // draw center baseline
  ctx.strokeStyle = '#e0e7ef';
  ctx.lineWidth   = 1;
  ctx.beginPath(); ctx.moveTo(0, MID); ctx.lineTo(W, MID); ctx.stroke();

  const drawn = Math.min(step, N_TOTAL);
  const xScale = W / N_TOTAL;

  function drawSignal(arr, color, lineW) {
    ctx.strokeStyle = color;
    ctx.lineWidth   = lineW;
    ctx.beginPath();
    for (let n = 0; n < drawn; n++) {
      const x = n * xScale;
      const y = MID - arr[n] * SCALE;
      n === 0 ? ctx.moveTo(x, y) : ctx.lineTo(x, y);
    }
    ctx.stroke();
  }

  if (signals) {
    drawSignal(signals.noisy, '#e63946', 1.2);
    drawSignal(signals.clean, '#457b9d', 1.5);
    if (output) drawSignal(output.slice(0, drawn), '#2a9d8f', 2);
  }
}

function animate() {
  if (step < N_TOTAL) {
    step += 8;  // advance 8 samples per animation frame for smooth visual
    drawFrame();
    const pct = Math.min(100, Math.round(step * 100 / N_TOTAL));
    statusEl.textContent = i18nFmt('Filtering… %pct%%', { pct });
    statusEl.className = 'status running';
    rafId = requestAnimationFrame(animate);
  } else {
    drawFrame();
    statusEl.textContent = 'Done — the hum is gone. Try a larger μ for faster (but noisier) convergence.';
    statusEl.className = 'status done';
    runBtn.textContent  = 'Run';
    runBtn.disabled     = false;
  }
}

function start() {
  if (rafId) { cancelAnimationFrame(rafId); rafId = null; }
  step    = 0;
  signals = genSignals();
  const mu = parseFloat(muInput.value);
  output  = runLMS(signals.ref, signals.noisy, mu);
  runBtn.textContent = 'Running…';
  runBtn.disabled    = true;
  animate();
}

function reset() {
  if (rafId) { cancelAnimationFrame(rafId); rafId = null; }
  step    = 0;
  signals = null;
  output  = null;
  ctx.clearRect(0, 0, W, H);
  // draw center baseline
  ctx.strokeStyle = '#e0e7ef';
  ctx.lineWidth   = 1;
  ctx.beginPath(); ctx.moveTo(0, MID); ctx.lineTo(W, MID); ctx.stroke();
  statusEl.textContent = 'Press Run to start the LMS filter.';
  statusEl.className   = 'status';
  runBtn.textContent   = 'Run';
  runBtn.disabled      = false;
}

runBtn.addEventListener('click', start);
resetBtn.addEventListener('click', reset);

// draw the empty baseline on page load
ctx.strokeStyle = '#e0e7ef';
ctx.lineWidth   = 1;
ctx.beginPath(); ctx.moveTo(0, MID); ctx.lineTo(W, MID); ctx.stroke();

<!-- LMS adaptive noise cancellation demo: a filter learns the hum path and subtracts it. -->
<div class="controls">
  <label>Step size μ: <input id="mu" type="range" min="0.001" max="0.05" step="0.001" value="0.01"></label>
  <span id="mu-val">0.010</span>
  <button id="run-btn" type="button">Run</button>
  <button id="reset-btn" type="button" class="ghost">Reset</button>
</div>
<canvas id="cnv" width="560" height="280" aria-label="Signal waveforms — noisy input, clean signal, and LMS filter output"></canvas>
<div class="legend">
  <span class="leg-noisy">Noisy input</span>
  <span class="leg-clean">Clean signal</span>
  <span class="leg-output">LMS output</span>
</div>
<div class="status" id="status">Press Run to start the LMS filter.</div>

* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; margin: 0; color: #222; }
.controls { display: flex; align-items: center; gap: .7rem; flex-wrap: wrap; margin-bottom: .6rem; }
label { font-size: .9rem; display: flex; align-items: center; gap: .4rem; }
input[type=range] { width: 120px; }
#mu-val { font-size: .9rem; font-weight: 700; width: 3.5ch; }
button { font: 600 14px system-ui, sans-serif; padding: .45rem .9rem;
         border: 1px solid #1d3557; background: #1d3557; color: #fff;
         border-radius: 8px; cursor: pointer; }
button.ghost { background: #fff; color: #1d3557; }
canvas { display: block; border: 1px solid #cdd9e3; border-radius: 8px;
         width: 100%; max-width: 560px; background: #f8fafc; }
.legend { display: flex; gap: 1rem; font-size: .82rem; margin-top: .4rem; flex-wrap: wrap; }
.legend span { display: flex; align-items: center; gap: .3rem; }
.leg-noisy::before { content: ""; display: inline-block; width: 20px; height: 3px; background: #e63946; border-radius: 2px; }
.leg-clean::before  { content: ""; display: inline-block; width: 20px; height: 3px; background: #457b9d; border-radius: 2px; }
.leg-output::before { content: ""; display: inline-block; width: 20px; height: 3px; background: #2a9d8f; border-radius: 2px; }
.status { font-size: .95rem; font-weight: 600; margin-top: .5rem; min-height: 1.4em; color: #444; }
.status.running { color: #0a7d33; }
.status.done    { color: #0a7d33; }

const i18nFmt = (s, v) => s.replace(/%(\w+)%/g, (_, k) => String(v[k] ?? ''));

// Simulation parameters
const SAMPLE_RATE = 200;   // samples per second (conceptual — actual playback is visual)
const N_TOTAL    = 600;    // total samples to process
const L          = 8;      // FIR filter taps (weights): longer = better at complex paths, slower
const HUM_FREQ   = 0.25;   // normalized hum frequency (50 Hz / 200 Hz sample rate = 0.25)
const SIG_FREQ   = 0.07;   // normalized signal frequency (speech-like low freq)
const SIG_START  = 100;    // sample where the signal burst starts
const SIG_END    = 500;    // sample where the signal burst ends
const HUM_AMP    = 1.0;
const SIG_AMP    = 0.6;

// Generate the three signal arrays: reference hum, clean signal, noisy mix.
function genSignals() {
  const ref    = new Float64Array(N_TOTAL);
  const clean  = new Float64Array(N_TOTAL);
  const noisy  = new Float64Array(N_TOTAL);
  const delay  = 3;   // the hum picked up at the signal mic is delayed + phase-shifted vs. the reference
  for (let n = 0; n < N_TOTAL; n++) {
    ref[n]   = HUM_AMP * Math.sin(2 * Math.PI * HUM_FREQ * n);
    const sig = (n >= SIG_START && n < SIG_END)
                ? SIG_AMP * Math.sin(2 * Math.PI * SIG_FREQ * n) : 0;
    clean[n] = sig;
    const hum = HUM_AMP * Math.sin(2 * Math.PI * HUM_FREQ * (n - delay) + 0.3);
    noisy[n] = sig + hum;
  }
  return { ref, clean, noisy };
}

// Run the LMS adaptive filter over all samples and return the output (error) signal.
function runLMS(ref, noisy, mu) {
  const w   = new Float64Array(L);
  const out = new Float64Array(N_TOTAL);
  for (let n = 0; n < N_TOTAL; n++) {
    // Build the reference input vector x(n) = [ref(n), ref(n-1), ..., ref(n-L+1)]
    const x = new Float64Array(L);
    for (let k = 0; k < L; k++) x[k] = n - k >= 0 ? ref[n - k] : 0;
    // Compute filter output y(n) = w^T * x(n)
    let y = 0;
    for (let k = 0; k < L; k++) y += w[k] * x[k];
    const e = noisy[n] - y;  // error = desired - filter output; this is also the cleaned signal
    out[n] = e;
    // LMS weight update: w(n+1) = w(n) + 2*mu*e(n)*x(n)
    for (let k = 0; k < L; k++) w[k] += 2 * mu * e * x[k];
  }
  return out;
}

const canvas  = document.getElementById('cnv');
const ctx     = canvas.getContext('2d');
const muInput = document.getElementById('mu');
const muVal   = document.getElementById('mu-val');
const runBtn  = document.getElementById('run-btn');
const resetBtn= document.getElementById('reset-btn');
const statusEl= document.getElementById('status');

muInput.addEventListener('input', () => { muVal.textContent = parseFloat(muInput.value).toFixed(3); });

let rafId = null;
let step  = 0;
let signals = null;
let output  = null;
const W = 560, H = 280;
const MID = H / 2;
const SCALE = 80; // amplitude to pixels

function drawFrame() {
  ctx.clearRect(0, 0, W, H);
  // draw center baseline
  ctx.strokeStyle = '#e0e7ef';
  ctx.lineWidth   = 1;
  ctx.beginPath(); ctx.moveTo(0, MID); ctx.lineTo(W, MID); ctx.stroke();

  const drawn = Math.min(step, N_TOTAL);
  const xScale = W / N_TOTAL;

  function drawSignal(arr, color, lineW) {
    ctx.strokeStyle = color;
    ctx.lineWidth   = lineW;
    ctx.beginPath();
    for (let n = 0; n < drawn; n++) {
      const x = n * xScale;
      const y = MID - arr[n] * SCALE;
      n === 0 ? ctx.moveTo(x, y) : ctx.lineTo(x, y);
    }
    ctx.stroke();
  }

  if (signals) {
    drawSignal(signals.noisy, '#e63946', 1.2);
    drawSignal(signals.clean, '#457b9d', 1.5);
    if (output) drawSignal(output.slice(0, drawn), '#2a9d8f', 2);
  }
}

function animate() {
  if (step < N_TOTAL) {
    step += 8;  // advance 8 samples per animation frame for smooth visual
    drawFrame();
    const pct = Math.min(100, Math.round(step * 100 / N_TOTAL));
    statusEl.textContent = i18nFmt('Filtering… %pct%%', { pct });
    statusEl.className = 'status running';
    rafId = requestAnimationFrame(animate);
  } else {
    drawFrame();
    statusEl.textContent = 'Done — the hum is gone. Try a larger μ for faster (but noisier) convergence.';
    statusEl.className = 'status done';
    runBtn.textContent  = 'Run';
    runBtn.disabled     = false;
  }
}

function start() {
  if (rafId) { cancelAnimationFrame(rafId); rafId = null; }
  step    = 0;
  signals = genSignals();
  const mu = parseFloat(muInput.value);
  output  = runLMS(signals.ref, signals.noisy, mu);
  runBtn.textContent = 'Running…';
  runBtn.disabled    = true;
  animate();
}

function reset() {
  if (rafId) { cancelAnimationFrame(rafId); rafId = null; }
  step    = 0;
  signals = null;
  output  = null;
  ctx.clearRect(0, 0, W, H);
  // draw center baseline
  ctx.strokeStyle = '#e0e7ef';
  ctx.lineWidth   = 1;
  ctx.beginPath(); ctx.moveTo(0, MID); ctx.lineTo(W, MID); ctx.stroke();
  statusEl.textContent = 'Press Run to start the LMS filter.';
  statusEl.className   = 'status';
  runBtn.textContent   = 'Run';
  runBtn.disabled      = false;
}

runBtn.addEventListener('click', start);
resetBtn.addEventListener('click', reset);

// draw the empty baseline on page load
ctx.strokeStyle = '#e0e7ef';
ctx.lineWidth   = 1;
ctx.beginPath(); ctx.moveTo(0, MID); ctx.lineTo(W, MID); ctx.stroke();

Result

Console

Notice: a larger $\mu$ converges faster but risks overshooting and becoming unstable. A smaller $\mu$ is rock-solid but slow. The sweet spot is the art — and the math — of adaptive filtering.

The Real Complexity

The goal of the filter is to minimize the mean squared error $E[e^2(n)]$ , where the error is:

e(n) = d(n) - \mathbf{w}^T(n)\,\mathbf{x}(n)

Here $d(n)$ is the desired signal (contaminated microphone), $\mathbf{x}(n)$ is the reference noise vector, and $\mathbf{w}(n)$ is the filter weight vector of length $L$ .

True gradient descent on this surface would require computing expectations — expensive and off-line. The LMS trick is to use the instantaneous gradient instead, replacing the expectation with a single sample:

\mathbf{w}(n+1) = \mathbf{w}(n) + 2\mu\, e(n)\, \mathbf{x}(n)

This is stochastic gradient descent applied to filtering, and it is $O(L)$ per sample — the same cost as running the filter itself.

Convergence is guaranteed when:

0 < \mu < \frac{1}{\lambda_{\max}}

where $\lambda_{\max}$ is the largest eigenvalue of the input autocorrelation matrix $\mathbf{R} = E[\mathbf{x}(n)\mathbf{x}^T(n)]$ . In practice, $\mu < 1/(\text{tr}(\mathbf{R}))$ is a safe bound that can be estimated from the signal power. The speed of convergence scales as $O(1 - 2\mu\lambda_{\min})$ ; choosing $\mu$ well is the central engineering decision.

Variants like NLMS (normalized LMS) divide the update by $\|\mathbf{x}(n)\|^2$ , making it invariant to input power — a major practical improvement. RLS (Recursive Least Squares) converges in as few as $L$ steps but costs $O(L^2)$ per sample. For real-time audio on constrained hardware, LMS remains the dominant choice because of its simplicity and efficiency.

Where It Matters

"Track a correlated interference and subtract it" turns out to be one of the most broadly useful primitives in signal processing:

Noise-cancelling headphones: the ear-cup microphone picks up ambient noise; the LMS filter models the path to the speaker and injects the anti-noise before you hear it.
Telephone echo cancellation: your voice travels to the far end and leaks back through the loudspeaker; an adaptive filter models the echo path and removes it from the return channel.
ECG / EEG denoising: powerline hum at 50 or 60 Hz corrupts biomedical signals; a reference electrode near the power cable feeds an LMS filter that strips the interference while leaving brain or heart signals intact.
Active vibration control: accelerometers on an engine feed reference signals into LMS filters that drive actuators to cancel structural vibration in aircraft cabins and car bodies.
Acoustic feedback cancellation: hearing aids and PA systems use adaptive filters to prevent the squeal caused when amplified sound re-enters the microphone.

The algorithm also connects deeply to the theory studied elsewhere on this site: the LMS update is stochastic gradient descent — the same engine that trains neural networks. The only difference is scale: LMS updates a handful of weights in microseconds; a neural network updates millions over hours.

Conclusion

Adaptive noise cancellation is one of the few complete victories in applied mathematics. The problem — remove correlated interference without knowing the signal — looked daunting, but Widrow and Hoff's 1960 LMS rule reduced it to a single line: nudge the weights toward smaller error, repeat every microsecond. Convergence is provable, cost is $O(L)$ per sample, and the algorithm fits on hardware with no floating-point unit.

What makes it especially satisfying is the feedback loop: the filter uses its own error to teach itself. Every sample is both a measurement and a lesson. By the time the hum has played for a second, the filter has already learned to erase it — and keeps re-learning as the noise path drifts with temperature, movement, or load. The hum never had a chance.

Comments

Loading comments...

https://www.kipuhub.com/en/article/adaptive-noise-cancellation/Content licensed under CC BY-NC 4.0.

Cookies & privacy

Adaptive Noise Cancellation

How a self-adjusting filter learns to subtract the hum