Introduction

You type a perfectly valid equation into a computer and get back a number that is completely wrong. The algorithm ran without error. The code has no bug. Yet the answer is garbage.

This happens because some problems are ill-conditioned: they are so sensitive to tiny perturbations in the input that unavoidable rounding errors — the kind introduced every time a computer stores a real number in finite bits — get amplified into enormous errors in the output.

The condition number of a problem is a single value that measures this amplification. A condition number near 1 is healthy: small errors stay small. A condition number in the millions means that even a rounding error at the fifteenth decimal place can corrupt every significant digit of your answer.

Ill-conditioning is not a bug you can fix by switching to a better algorithm. It is a property of the problem itself — the mathematical object you are asking about, not the method you use to study it.

Try It

The slider below controls how close a 2×22 \times 2 linear system gets to singular (i.e., how large its condition number grows). A tiny perturbation ε\varepsilon is added to the right-hand side, and you can see how much that error grows in the solution.

<!-- Condition number demo: a 2x2 linear system tuned to approach singular -->
<p class="hint">Drag the slider to make the matrix nearly singular. A tiny perturbation ε is added to the right-hand side — watch how the output error grows as the condition number rises.</p>
<div class="slider-row">
  <label for="kSlider">Condition number κ: <strong id="kappaVal">1</strong></label>
  <input type="range" id="kSlider" min="0" max="100" value="0" step="1">
</div>
<div class="panels">
  <div class="panel">
    <div class="panel-title">Linear system Ax = b</div>
    <div id="sysDisplay" class="matrix-display"></div>
  </div>
  <div class="panel">
    <div class="panel-title">Exact solution</div>
    <div id="exactDisplay" class="vec-display"></div>
  </div>
  <div class="panel">
    <div class="panel-title">Perturbed solution (b + ε)</div>
    <div id="pertDisplay" class="vec-display"></div>
  </div>
</div>
<div class="panels">
  <div class="panel">
    <div class="panel-title">Condition number κ</div>
    <div id="kappaDisplay" class="number-big"></div>
  </div>
  <div class="panel">
    <div class="panel-title">Input error (relative)</div>
    <div id="inErrDisplay" class="number-big"></div>
  </div>
  <div class="panel">
    <div class="panel-title">Output error (relative)</div>
    <div id="outErrDisplay" class="number-big good"></div>
  </div>
</div>
<div class="status" id="status"></div>
<button id="resetBtn" type="button" class="ghost">Reset</button>

/* Layout and color styles for the condition number demo */
* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; color: #222; margin: 0; padding: 14px; }
.hint { font-size: .88rem; color: #444; margin: 0 0 .8rem; line-height: 1.5; }
.slider-row { display: flex; align-items: center; gap: .8rem; margin-bottom: .9rem; flex-wrap: wrap; }
.slider-row label { font-size: .9rem; white-space: nowrap; }
#kSlider { flex: 1; min-width: 120px; accent-color: #1d3557; }
.panels { display: flex; gap: .5rem; margin-bottom: .5rem; flex-wrap: wrap; }
.panel { flex: 1; min-width: 100px; background: #e8eef3; border: 1px solid #cdd9e3; border-radius: 8px; padding: .5rem .7rem; }
.panel-title { font-size: .72rem; font-weight: 700; color: #5a7088; text-transform: uppercase; letter-spacing: .04em; margin-bottom: .3rem; }
.matrix-display { font: 600 .78rem ui-monospace, monospace; color: #1d3557; line-height: 1.7; white-space: pre; }
.vec-display { font: 600 .78rem ui-monospace, monospace; color: #1d3557; line-height: 1.7; white-space: pre; }
.number-big { font: 700 1.1rem ui-monospace, monospace; color: #1d3557; padding-top: .1rem; }
.number-big.good { color: #0a7d33; }
.number-big.warn { color: #c97a00; }
.number-big.bad { color: #c92f3c; }
.status { font-size: .9rem; font-weight: 600; min-height: 1.4em; margin: .4rem 0; }
.status.ok { color: #0a7d33; }
.status.warn { color: #c97a00; }
.status.bad { color: #c92f3c; }
button.ghost { font: 600 13px system-ui, sans-serif; padding: .4rem .8rem; border: 1px solid #1d3557; background: #fff; color: #1d3557; border-radius: 8px; cursor: pointer; }

const i18nFmt = (s, v) => s.replace(/%(\w+)%/g, (_, k) => String(v[k] ?? ''));

// A(t) = [[1, 1-t], [1, 1+t]] with true solution x = [1, 0]^T
// A(t) = [[1, 1-t], [1, 1+t]]  with true solution x = [1, 0]^T
// b is computed from A and the true solution so the exact answer is always x=[1,0]
// As t->0, A becomes [[1,1],[1,1]] which is singular (kappa->infinity).
// A perturbation EPS is added to b[0] to simulate a tiny measurement error
const EPS = 1e-7; // size of the input perturbation (similar to machine epsilon)

function buildSystem(t) {
  // Build matrix A and right-hand side b for parameter t
  const A = [[1, 1 - t], [1, 1 + t]];
  const b = [A[0][0] * 1 + A[0][1] * 0, A[1][0] * 1 + A[1][1] * 0];
  return { A, b };
}

function solve2x2(A, b) {
  // Solve 2x2 linear system by Cramer's rule
  const det = A[0][0] * A[1][1] - A[0][1] * A[1][0];
  if (Math.abs(det) < 1e-15) return null;
  const x0 = (b[0] * A[1][1] - b[1] * A[0][1]) / det;
  const x1 = (A[0][0] * b[1] - A[1][0] * b[0]) / det;
  return [x0, x1];
}

function condNumber(A) {
  // Condition number via SVD formula for 2x2 matrices
  // For 2x2: sigma_max and sigma_min via SVD shortcut
  const a = A[0][0], b = A[0][1], c = A[1][0], d = A[1][1];
  const s1 = a * a + b * b + c * c + d * d;
  const s2 = Math.sqrt(Math.pow(a * a + b * b - c * c - d * d, 2) + 4 * Math.pow(a * c + b * d, 2));
  const sigMax = Math.sqrt((s1 + s2) / 2);
  const sigMin = Math.sqrt(Math.max(0, (s1 - s2) / 2));
  if (sigMin < 1e-15) return Infinity;
  return sigMax / sigMin;
}

function fmt2(n) {
  // Format a number for display
  if (!isFinite(n)) return '∞';
  if (Math.abs(n) >= 1e6) return n.toExponential(2);
  return n.toFixed(n < 0.01 && n > -0.01 ? 6 : 4);
}

function update() {
  const slider = document.getElementById('kSlider');
  // Map slider (0-100) to t in (0, 0.5] so kappa grows smoothly to very large values
  const t = 0.5 * Math.pow(1 - slider.value / 100, 3);
  const { A, b } = buildSystem(t);
  const kappa = condNumber(A);

  // Solve exact and perturbed systems
  const bPerturb = [b[0] + EPS, b[1]];
  const xExact = solve2x2(A, b);
  const xPerturb = solve2x2(A, bPerturb);

  // Compute relative input and output errors
  const inErr = EPS / Math.sqrt(b[0] * b[0] + b[1] * b[1]);
  let outErr = null;
  if (xExact && xPerturb) {
    const dx0 = xPerturb[0] - xExact[0], dx1 = xPerturb[1] - xExact[1];
    const xNorm = Math.sqrt(xExact[0] * xExact[0] + xExact[1] * xExact[1]);
    outErr = Math.sqrt(dx0 * dx0 + dx1 * dx1) / (xNorm || 1);
  }

  // Update condition number displays
  document.getElementById('kappaVal').textContent = isFinite(kappa) ? Math.round(kappa).toLocaleString() : '∞';
  document.getElementById('kappaDisplay').textContent = isFinite(kappa) ? Math.round(kappa).toLocaleString() : '∞';

  // Update system display
  document.getElementById('sysDisplay').textContent =
    `[${fmt2(A[0][0])}  ${fmt2(A[0][1])}] [x₀]   [${fmt2(b[0])}]\n` +
    `[${fmt2(A[1][0])}  ${fmt2(A[1][1])}] [x₁] = [${fmt2(b[1])}]`;

  // Update solution displays
  if (xExact) {
    document.getElementById('exactDisplay').textContent =
      `x₀ = ${fmt2(xExact[0])}\nx₁ = ${fmt2(xExact[1])}`;
  }
  if (xPerturb) {
    document.getElementById('pertDisplay').textContent =
      `x₀ = ${fmt2(xPerturb[0])}\nx₁ = ${fmt2(xPerturb[1])}`;
  }

  // Update error displays with color coding
  document.getElementById('inErrDisplay').textContent = inErr.toExponential(2);

  const outEl = document.getElementById('outErrDisplay');
  if (outErr !== null) {
    outEl.textContent = outErr.toExponential(2);
    if (outErr < 1e-4) { outEl.className = 'number-big good'; }
    else if (outErr < 1e-1) { outEl.className = 'number-big warn'; }
    else { outEl.className = 'number-big bad'; }
  }

  // Update status message
  const statusEl = document.getElementById('status');
  if (!isFinite(kappa) || kappa > 1e10) {
    statusEl.textContent = '⚠ Nearly singular — solution meaningless!';
    statusEl.className = 'status bad';
  } else if (kappa > 1e5) {
    statusEl.textContent = i18nFmt('⚠ Ill-conditioned (κ = %kappa%) — large error amplification.', { kappa: Math.round(kappa).toLocaleString() });
    statusEl.className = 'status warn';
  } else if (kappa > 100) {
    statusEl.textContent = i18nFmt('△ Moderately conditioned (κ = %kappa%) — some amplification.', { kappa: Math.round(kappa).toLocaleString() });
    statusEl.className = 'status warn';
  } else {
    statusEl.textContent = i18nFmt('✓ Well-conditioned (κ = %kappa%) — small error amplification.', { kappa: Math.round(kappa).toLocaleString() });
    statusEl.className = 'status ok';
  }
}

document.getElementById('kSlider').addEventListener('input', update);
document.getElementById('resetBtn').addEventListener('click', () => {
  document.getElementById('kSlider').value = 0;
  update();
});
update();
Result
Console

Notice what happens as the matrix approaches singular: a perturbation that is invisible in the input — smaller than a typical rounding error — produces an error in the solution that is many times larger. This ratio is exactly the condition number at work.

The Real Complexity

For a matrix AA, the condition number is defined as

κ(A)=∥A∥⋅∥A−1∥\kappa(A) = \|A\| \cdot \|A^{-1}\|

In the common case of the 2-norm, this equals the ratio of the largest singular value σmax⁡\sigma_{\max} to the smallest σmin⁡\sigma_{\min}:

κ(A)=σmax⁡σmin⁡\kappa(A) = \frac{\sigma_{\max}}{\sigma_{\min}}

When σmin⁡\sigma_{\min} approaches zero — meaning the matrix is nearly singular — κ(A)\kappa(A) shoots toward infinity.

Why it sets a hard floor on accuracy. If your computer arithmetic works to pp decimal digits of precision, a system with condition number κ\kappa can lose up to log⁡10κ\log_{10} \kappa digits to amplified rounding error. So a system with κ=108\kappa = 10^{8} on a standard 15-digit machine may deliver only about 7 correct digits — and nothing about the algorithm can recover the lost digits, because the information was destroyed at the input level.

It is a property of the problem, not the solver. You can swap Gaussian elimination for a fancier iterative method, but if κ\kappa is large, every method will suffer. The only real cures are:

  • Reformulate the problem so the underlying matrix has a lower condition number.
  • Preconditioning: multiply both sides of the system by a matrix P−1P^{-1} chosen so that P−1AP^{-1}A has a much smaller condition number.
  • Use higher precision arithmetic to push the floor down, buying back some digits.

Connection to floating-point arithmetic. Floating-point numbers have a relative error floor called machine epsilon εmach\varepsilon_{\text{mach}}. The best bound on relative error in the solution of Ax=bAx = b is roughly κ(A)⋅εmach\kappa(A) \cdot \varepsilon_{\text{mach}}, so a large κ\kappa directly multiplies whatever imprecision the hardware introduces.

Where It Matters

Almost every field that uses computers to solve equations must reckon with condition numbers:

  • Structural engineering (FEM): finite-element models of buildings or bridges produce huge sparse linear systems. Engineers check the condition number before trusting a simulation, because an ill-conditioned stiffness matrix can make a structure look stable on screen when it is actually at risk.
  • Machine learning: the Hessian of a loss function during training has a condition number that directly controls how fast gradient descent converges. Poorly conditioned Hessians are the root cause of slow training and the reason techniques like batch normalization were invented.
  • Medical imaging (MRI, CT): reconstructing an image from measurements is an inverse problem. Ill-conditioning means that measurement noise gets amplified into image artifacts, which is why regularization methods such as Tikhonov regularization are standard.
  • PageRank and link analysis: Google's original ranking algorithm solves a large eigenvalue problem. Near-singular cases arise when parts of the web graph are almost disconnected, and damping factors were introduced partly to keep the system well-conditioned.
  • Cryptography: deliberate ill-conditioning is exploited in some lattice-based cryptosystems — solving a nearly-singular lattice problem is easy with the right key and apparently intractable without it.

See also dimensionality reduction, which uses singular value decomposition and must carefully handle small singular values — exactly the source of ill-conditioning.

Conclusion

The condition number is one of those rare concepts that changes how you read a numerical answer. A result that looks precise to ten decimal places might carry only two significant digits if the underlying problem is ill-conditioned. And there is no clever algorithm that can recover what the ill-conditioning has already destroyed.

The sobering lesson is that error amplification is a property of the problem, not the method. Before trusting any large-scale computation — a structural simulation, a trained model, a medical image reconstruction — it is worth asking: what is the condition number here? If the answer is large, the computation deserves skepticism, extra precision, or a smarter problem reformulation.

Numerical stability is not a detail to leave to library authors. It is a fundamental question about whether the mathematics you wrote down can even be answered reliably by a finite machine.

Share this article

Pick a channel — or use your device's native share sheet.

Comments

Loading comments...

https://www.kipuhub.com/en/article/condition-number/Content licensed under CC BY-NC 4.0.