The Condition Number

You type a perfectly valid equation into a computer and get back a number that is completely wrong. The algorithm ran without error. The code has no bug. Yet the answer is garbage.

This happens because some problems are ill-conditioned: they are so sensitive to tiny perturbations in the input that unavoidable rounding errors — the kind introduced every time a computer stores a real number in finite bits — get amplified into enormous errors in the output.

The condition number of a problem is a single value that measures this amplification. A condition number near 1 is healthy: small errors stay small. A condition number in the millions means that even a rounding error at the fifteenth decimal place can corrupt every significant digit of your answer.

Ill-conditioning is not a bug you can fix by switching to a better algorithm. It is a property of the problem itself — the mathematical object you are asking about, not the method you use to study it.

The slider below controls how close a $2 \times 2$ linear system gets to singular (i.e., how large its condition number grows). A tiny perturbation $\varepsilon$ is added to the right-hand side, and you can see how much that error grows in the solution.

<!-- {{c_demo_intro}} -->
<p class="hint">{{hint_para}}</p>
<div class="slider-row">
  <label for="kSlider">{{label_kappa}}: <strong id="kappaVal">1</strong></label>
  <input type="range" id="kSlider" min="0" max="100" value="0" step="1">
</div>
<div class="panels">
  <div class="panel">
    <div class="panel-title">{{panel_system}}</div>
    <div id="sysDisplay" class="matrix-display"></div>
  </div>
  <div class="panel">
    <div class="panel-title">{{panel_exact}}</div>
    <div id="exactDisplay" class="vec-display"></div>
  </div>
  <div class="panel">
    <div class="panel-title">{{panel_perturbed}}</div>
    <div id="pertDisplay" class="vec-display"></div>
  </div>
</div>
<div class="panels">
  <div class="panel">
    <div class="panel-title">{{panel_kappa_label}}</div>
    <div id="kappaDisplay" class="number-big"></div>
  </div>
  <div class="panel">
    <div class="panel-title">{{panel_input_error}}</div>
    <div id="inErrDisplay" class="number-big"></div>
  </div>
  <div class="panel">
    <div class="panel-title">{{panel_output_error}}</div>
    <div id="outErrDisplay" class="number-big good"></div>
  </div>
</div>
<div class="status" id="status"></div>
<button id="resetBtn" type="button" class="ghost">{{btn_reset}}</button>

/* {{c_layout}} */
* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; color: #222; margin: 0; padding: 14px; }
.hint { font-size: .88rem; color: #444; margin: 0 0 .8rem; line-height: 1.5; }
.slider-row { display: flex; align-items: center; gap: .8rem; margin-bottom: .9rem; flex-wrap: wrap; }
.slider-row label { font-size: .9rem; white-space: nowrap; }
#kSlider { flex: 1; min-width: 120px; accent-color: #1d3557; }
.panels { display: flex; gap: .5rem; margin-bottom: .5rem; flex-wrap: wrap; }
.panel { flex: 1; min-width: 100px; background: #e8eef3; border: 1px solid #cdd9e3; border-radius: 8px; padding: .5rem .7rem; }
.panel-title { font-size: .72rem; font-weight: 700; color: #5a7088; text-transform: uppercase; letter-spacing: .04em; margin-bottom: .3rem; }
.matrix-display { font: 600 .78rem ui-monospace, monospace; color: #1d3557; line-height: 1.7; white-space: pre; }
.vec-display { font: 600 .78rem ui-monospace, monospace; color: #1d3557; line-height: 1.7; white-space: pre; }
.number-big { font: 700 1.1rem ui-monospace, monospace; color: #1d3557; padding-top: .1rem; }
.number-big.good { color: #0a7d33; }
.number-big.warn { color: #c97a00; }
.number-big.bad { color: #c92f3c; }
.status { font-size: .9rem; font-weight: 600; min-height: 1.4em; margin: .4rem 0; }
.status.ok { color: #0a7d33; }
.status.warn { color: #c97a00; }
.status.bad { color: #c92f3c; }
button.ghost { font: 600 13px system-ui, sans-serif; padding: .4rem .8rem; border: 1px solid #1d3557; background: #fff; color: #1d3557; border-radius: 8px; cursor: pointer; }

// Code not found

Notice what happens as the matrix approaches singular: a perturbation that is invisible in the input — smaller than a typical rounding error — produces an error in the solution that is many times larger. This ratio is exactly the condition number at work.

For a matrix $A$, the condition number is defined as

$$\kappa(A) = \|A\| \cdot \|A^{-1}\|$$

In the common case of the 2-norm, this equals the ratio of the largest singular value $\sigma_{\max}$ to the smallest $\sigma_{\min}$:

$$\kappa(A) = \frac{\sigma_{\max}}{\sigma_{\min}}$$

When $\sigma_{\min}$ approaches zero — meaning the matrix is nearly singular — $\kappa(A)$ shoots toward infinity.

Why it sets a hard floor on accuracy. If your computer arithmetic works to $p$ decimal digits of precision, a system with condition number $\kappa$ can lose up to $\log_{10} \kappa$ digits to amplified rounding error. So a system with $\kappa = 10^{8}$ on a standard 15-digit machine may deliver only about 7 correct digits — and nothing about the algorithm can recover the lost digits, because the information was destroyed at the input level.

It is a property of the problem, not the solver. You can swap Gaussian elimination for a fancier iterative method, but if $\kappa$ is large, every method will suffer. The only real cures are:

• Reformulate the problem so the underlying matrix has a lower condition number.
• Preconditioning: multiply both sides of the system by a matrix $P^{-1}$ chosen so that $P^{-1}A$ has a much smaller condition number.
• Use higher precision arithmetic to push the floor down, buying back some digits.

Connection to floating-point arithmetic. Floating-point numbers have a relative error floor called machine epsilon $\varepsilon_{\text{mach}}$. The best bound on relative error in the solution of $Ax = b$ is roughly $\kappa(A) \cdot \varepsilon_{\text{mach}}$, so a large $\kappa$ directly multiplies whatever imprecision the hardware introduces.

Almost every field that uses computers to solve equations must reckon with condition numbers:

• Structural engineering (FEM): finite-element models of buildings or bridges produce huge sparse linear systems. Engineers check the condition number before trusting a simulation, because an ill-conditioned stiffness matrix can make a structure look stable on screen when it is actually at risk.
• Machine learning: the Hessian of a loss function during training has a condition number that directly controls how fast gradient descent converges. Poorly conditioned Hessians are the root cause of slow training and the reason techniques like batch normalization were invented.
• Medical imaging (MRI, CT): reconstructing an image from measurements is an inverse problem. Ill-conditioning means that measurement noise gets amplified into image artifacts, which is why regularization methods such as Tikhonov regularization are standard.
• PageRank and link analysis: Google's original ranking algorithm solves a large eigenvalue problem. Near-singular cases arise when parts of the web graph are almost disconnected, and damping factors were introduced partly to keep the system well-conditioned.
• Cryptography: deliberate ill-conditioning is exploited in some lattice-based cryptosystems — solving a nearly-singular lattice problem is easy with the right key and apparently intractable without it.

See also dimensionality reduction, which uses singular value decomposition and must carefully handle small singular values — exactly the source of ill-conditioning.

The condition number is one of those rare concepts that changes how you read a numerical answer. A result that looks precise to ten decimal places might carry only two significant digits if the underlying problem is ill-conditioned. And there is no clever algorithm that can recover what the ill-conditioning has already destroyed.

The sobering lesson is that error amplification is a property of the problem, not the method. Before trusting any large-scale computation — a structural simulation, a trained model, a medical image reconstruction — it is worth asking: what is the condition number here? If the answer is large, the computation deserves skepticism, extra precision, or a smarter problem reformulation.

Numerical stability is not a detail to leave to library authors. It is a fundamental question about whether the mathematics you wrote down can even be answered reliably by a finite machine.