Introduction

Imagine you are describing a location in a city. You say: "three blocks north, two blocks east." Those two directions — north and east — are orthogonal: they share nothing, they are independent, and together they span the whole plane. Most coordinate systems we use in practice share this pleasant property.

But raw data rarely arrives in such a tidy form. A dataset of measurements might give you vectors that point in similar directions, partially overlapping. To do geometry, solve equations, or train a machine-learning model, you almost always need a cleaner basis — one where the vectors are perpendicular to each other (orthogonal) and each has length one (normal). Together that is called an orthonormal basis.

The Gram-Schmidt process, developed independently by Jørgen Pedersen Gram (1883) and Erhard Schmidt (1907), is the classical recipe for building exactly that. The idea is beautifully simple: take each vector in turn, subtract the shadow it casts onto all previous vectors, and what remains is a new direction that is perpendicular to everything before it. Normalize it to length one, and you have your next basis vector.

The process always works for any set of linearly independent vectors, and it produces an orthonormal basis for the subspace they span. But there is a catch: when implemented on a computer, the classical version can lose accuracy due to floating-point errors. A small tweak — the modified Gram-Schmidt algorithm — fixes this without changing the mathematical result.

Try It

The demo below shows two vectors in the plane. Drag the tip of each arrow to reposition it, then press Orthogonalize to run Gram-Schmidt step by step.

<!-- Gram-Schmidt interactive demo — two draggable vectors in the plane -->
<div class="hint">Drag the tips of the blue arrows to set two vectors, then press Orthogonalize. Watch Gram-Schmidt strip each vector of its overlap with the previous one.</div>
<canvas id="gs-canvas" width="380" height="280"></canvas>
<div class="status" id="gs-status">Drag the arrow tips, then press Orthogonalize.</div>
<div class="btns">
  <button id="btn-run" type="button">Orthogonalize</button>
  <button id="btn-reset" type="button" class="ghost">Reset</button>
</div>

/* Demo styles */
* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; color: #222; margin: 0; }
.hint { font-size: .88rem; color: #444; margin: 0 0 .6rem; line-height: 1.5; }
canvas { display: block; background: #f4f7fa; border-radius: 10px; border: 1px solid #dce3eb;
         cursor: crosshair; touch-action: none; max-width: 100%; }
.status { font-size: .95rem; font-weight: 600; margin: .45rem 0; min-height: 1.4em; }
.status.ok  { color: #0a7d33; }
.status.bad { color: #c92f3c; }
.status.info { color: #1d4e8f; }
.btns { display: flex; gap: .5rem; flex-wrap: wrap; }
button { font: 600 14px system-ui; padding: .45rem .9rem; border: 1px solid #1d3557;
         background: #1d3557; color: #fff; border-radius: 8px; cursor: pointer; }
button.ghost { background: #fff; color: #1d3557; }

const i18nFmt = (s, v) => s.replace(/%(\w+)%/g, (_, k) => String(v[k] ?? ''));
// Canvas dimensions
const canvas = document.getElementById('gs-canvas');
const ctx = canvas.getContext('2d');
const W = canvas.width, H = canvas.height;
// Origin is at the center of the canvas
const OX = W / 2, OY = H / 2;
const SCALE = 90;

// Current input vectors and computed basis vectors
let v1 = [1.4, 0.6];
let v2 = [0.5, 1.3];
let e1 = null, e2 = null;
let phase = 'init';

// Which vector tip is being dragged
let dragging = null;

function dot(a, b) { return a[0]*b[0] + a[1]*b[1]; }
function norm(a) { return Math.sqrt(dot(a, a)); }
function scale(a, s) { return [a[0]*s, a[1]*s]; }
function sub(a, b) { return [a[0]-b[0], a[1]-b[1]]; }
function add(a, b) { return [a[0]+b[0], a[1]+b[1]]; }
// Normalize a vector to unit length
function normalize(a) { const n = norm(a); return n < 1e-9 ? [1, 0] : scale(a, 1/n); }

function toScreen(v) { return [OX + v[0]*SCALE, OY - v[1]*SCALE]; }
function toWorld(px, py) { return [(px - OX)/SCALE, (OY - py)/SCALE]; }

// Draw a vector arrow with optional label and dashed style
function drawArrow(from, to, color, label, dashed) {
  const [fx, fy] = toScreen(from);
  const [tx, ty] = toScreen(to);
  ctx.save();
  ctx.strokeStyle = color; ctx.fillStyle = color;
  ctx.lineWidth = 2.2;
  if (dashed) ctx.setLineDash([5, 4]);
  ctx.beginPath(); ctx.moveTo(fx, fy); ctx.lineTo(tx, ty); ctx.stroke();
  ctx.setLineDash([]);
  // Draw the arrowhead at the tip
  const ang = Math.atan2(ty - fy, tx - fx);
  const hw = 8, hl = 14;
  ctx.beginPath();
  ctx.moveTo(tx, ty);
  ctx.lineTo(tx - hl*Math.cos(ang - 0.35), ty - hl*Math.sin(ang - 0.35));
  ctx.lineTo(tx - hl*Math.cos(ang + 0.35), ty - hl*Math.sin(ang + 0.35));
  ctx.closePath(); ctx.fill();
  if (label) {
    ctx.font = 'bold 13px system-ui';
    ctx.fillText(label, tx + 6, ty - 6);
  }
  ctx.restore();
}

// Draw a small right-angle symbol at the origin between two unit directions
function drawRightAngle(p, d1, d2, size) {
  const s = size / SCALE;
  const a = add(p, scale(d1, s));
  const b = add(add(p, scale(d1, s)), scale(d2, s));
  const c = add(p, scale(d2, s));
  const pts = [a, b, c].map(toScreen);
  ctx.save();
  ctx.strokeStyle = '#888'; ctx.lineWidth = 1.2;
  ctx.beginPath();
  ctx.moveTo(pts[0][0], pts[0][1]);
  ctx.lineTo(pts[1][0], pts[1][1]);
  ctx.lineTo(pts[2][0], pts[2][1]);
  ctx.stroke();
  ctx.restore();
}

function drawAxes() {
  ctx.save();
  ctx.strokeStyle = '#c8d4de'; ctx.lineWidth = 1;
  ctx.beginPath(); ctx.moveTo(0, OY); ctx.lineTo(W, OY); ctx.stroke();
  ctx.beginPath(); ctx.moveTo(OX, 0); ctx.lineTo(OX, H); ctx.stroke();
  ctx.restore();
}

// Clear canvas and redraw everything for the current phase
function draw() {
  ctx.clearRect(0, 0, W, H);
  drawAxes();
  const origin = [0, 0];

  // Draw the original input vectors v1 and v2
  drawArrow(origin, v1, '#1d3557', 'v₁', false);
  drawArrow(origin, v2, '#457b9d', 'v₂', false);

  if (phase === 'step1' || phase === 'done') {
    // Draw the first basis vector e1 (normalized v1)
    drawArrow(origin, e1, '#e63946', 'e₁', false);
  }
  if (phase === 'done') {
    // Draw the projection of v2 onto e1 (dashed)
    const proj = scale(e1, dot(v2, e1));
    drawArrow(origin, proj, '#f4a261', null, true);
    // Draw the second basis vector e2 (remainder after projection)
    drawArrow(origin, e2, '#2a9d8f', 'e₂', false);
    // Mark the right angle between e1 and e2 at the origin
    drawRightAngle(origin, e1, e2, 14);
  }

  // Draw draggable dots at the tips of v1 and v2
  [v1, v2].forEach((v, i) => {
    const [sx, sy] = toScreen(v);
    ctx.beginPath();
    ctx.arc(sx, sy, 6, 0, 2*Math.PI);
    ctx.fillStyle = i === 0 ? '#1d3557' : '#457b9d';
    ctx.fill();
  });
}

function setStatus(text, cls) {
  const el = document.getElementById('gs-status');
  el.textContent = text;
  el.className = 'status ' + (cls || '');
}

// Run one step of Gram-Schmidt on button click
function runGS() {
  if (phase === 'init' || phase === 'done') {
    // Step 1 — normalize v1 to get e1
    const n1 = norm(v1);
    if (n1 < 0.05) { setStatus('v1 is too short — drag its tip away from the origin.', 'bad'); return; }
    e1 = normalize(v1);
    phase = 'step1';
    draw();
    setStatus(i18nFmt('Step 1: e1 = normalize(v1). Original length was %n%.', { n: norm(v1).toFixed(3) }), 'info');
    return;
  }
  if (phase === 'step1') {
    // Step 2 — subtract projection of v2 onto e1, then normalize the remainder
    const proj = scale(e1, dot(v2, e1));
    const remainder = sub(v2, proj);
    const n2 = norm(remainder);
    if (n2 < 0.05) { setStatus('The vectors are parallel — e2 does not exist. Move v2 to a different direction.', 'bad'); return; }
    e2 = normalize(remainder);
    const d = Math.abs(dot(e1, e2));
    phase = 'done';
    draw();
    setStatus(i18nFmt('Step 2 done. |e1 · e2| = %d% (should be ≈ 0).', { d: d.toExponential(2) }), 'ok');
  }
}

// Reset vectors to initial positions and clear computed basis
function reset() {
  v1 = [1.4, 0.6];
  v2 = [0.5, 1.3];
  e1 = null; e2 = null;
  phase = 'init';
  draw();
  setStatus('Drag the arrow tips, then press Orthogonalize.', '');
}

// Pointer event handlers for dragging vector tips
function hitTest(px, py, v) {
  const [sx, sy] = toScreen(v);
  return Math.hypot(px - sx, py - sy) < 14;
}
canvas.addEventListener('pointerdown', e => {
  const r = canvas.getBoundingClientRect();
  const px = (e.clientX - r.left) * (W / r.width);
  const py = (e.clientY - r.top) * (H / r.height);
  if (hitTest(px, py, v1)) dragging = 'v1';
  else if (hitTest(px, py, v2)) dragging = 'v2';
});
canvas.addEventListener('pointermove', e => {
  if (!dragging) return;
  const r = canvas.getBoundingClientRect();
  const px = (e.clientX - r.left) * (W / r.width);
  const py = (e.clientY - r.top) * (H / r.height);
  const w = toWorld(px, py);
  if (dragging === 'v1') v1 = w; else v2 = w;
  phase = 'init'; e1 = null; e2 = null;
  draw();
  setStatus('Drag the arrow tips, then press Orthogonalize.', '');
});
canvas.addEventListener('pointerup', () => { dragging = null; });
canvas.addEventListener('pointercancel', () => { dragging = null; });

document.getElementById('btn-run').addEventListener('click', runGS);
document.getElementById('btn-reset').addEventListener('click', reset);
draw();
setStatus('Drag the arrow tips, then press Orthogonalize.', '');
Result
Console

Step 1 keeps v1\mathbf{v}_1 as-is and normalizes it to e1\mathbf{e}_1. Step 2 subtracts from v2\mathbf{v}_2 its projection onto e1\mathbf{e}_1 — whatever they share — leaving a remainder that is exactly perpendicular. Normalizing that remainder gives e2\mathbf{e}_2. The result is always a right angle, regardless of where you placed the original vectors — as long as they are not parallel.

The Real Complexity

Mathematically, classical and modified Gram-Schmidt are identical — both produce the same orthonormal basis. On a real computer, they behave very differently.

Classical Gram-Schmidt computes the full projection of vector vk\mathbf{v}_k onto all previous basis vectors at once:

uk=vk−∑j=1k−1⟨vk,ej⟩ ej\mathbf{u}_k = \mathbf{v}_k - \sum_{j=1}^{k-1} \langle \mathbf{v}_k, \mathbf{e}_j \rangle \, \mathbf{e}_j

When vectors are nearly parallel, uk\mathbf{u}_k is the difference of nearly equal numbers. Floating-point arithmetic then magnifies tiny rounding errors, and the resulting vectors can drift far from orthogonal. In bad cases, ⟨ei,ej⟩\langle \mathbf{e}_i, \mathbf{e}_j \rangle can be 10−610^{-6} or worse instead of zero.

Modified Gram-Schmidt subtracts one projection at a time, updating the working vector after each step:

uk(j)=uk(j−1)−⟨uk(j−1),ej⟩ ej\mathbf{u}_k^{(j)} = \mathbf{u}_k^{(j-1)} - \langle \mathbf{u}_k^{(j-1)}, \mathbf{e}_j \rangle \, \mathbf{e}_j

Each subtraction uses the most up-to-date version of the vector, so errors do not compound. The result is dramatically more orthogonal in practice.

Even modified Gram-Schmidt is not the last word. For the most demanding applications, Householder reflections (used in standard QR decompositions) provide even better numerical guarantees. But modified Gram-Schmidt remains the workhorse of iterative methods like GMRES and the Lanczos algorithm, where vectors arrive one at a time and you cannot afford to start over.

Where It Matters

Orthonormal bases are the backbone of numerical computation, and Gram-Schmidt is how you build them from scratch:

  • QR decomposition: the factorization A=QRA = QR (where QQ has orthonormal columns) underlies least-squares solvers, eigenvalue algorithms, and statistical regression. Gram-Schmidt is one of the three standard ways to compute QQ.
  • Principal Component Analysis (PCA): the principal components of a dataset must be orthogonal. After computing the covariance matrix, an eigenvector decomposition already produces orthogonal directions — but Gram-Schmidt is used in incremental and streaming variants where data arrives in batches.
  • Signal processing: orthonormal bases like the Fourier basis or wavelet bases let you represent signals without redundancy. Gram-Schmidt is the tool for building custom orthonormal frames tailored to a specific dataset.
  • Iterative linear solvers: Krylov-subspace methods (GMRES, Arnoldi, Lanczos) build an orthonormal basis for a growing subspace one vector at a time. Modified Gram-Schmidt keeps this basis numerically clean without recomputing everything from scratch.
  • Computer graphics: orthonormal coordinate frames are needed for camera orientations, surface normals, and tangent-space calculations. The Gram-Schmidt process (often a quick two-vector version) is a standard primitive in 3D engines.

Learn Gram-Schmidt and you have unlocked a tool that quietly runs inside nearly every serious numerical algorithm, from dimensionality reduction to differential equation solvers.

Conclusion

The Gram-Schmidt process distills a powerful idea into a single rule: subtract what two vectors share, keep what is new. Repeat that for each vector in your set, normalize at each step, and you are left with a family of directions that are perfectly perpendicular and independent — an orthonormal basis.

The classical version is easy to understand; the modified version is easy to run on a computer. That small reordering of subtractions, which changes nothing mathematically, is the difference between a basis that drifts and one that stays true.

The next time you solve a least-squares problem, run PCA on a dataset, or watch a CGI camera pivot smoothly, somewhere in that computation a set of vectors was straightened into right angles — and Gram-Schmidt, in one form or another, is why they ended up at exactly 90 degrees.

Share this article

Pick a channel — or use your device's native share sheet.

Comments

Loading comments...

https://www.kipuhub.com/en/article/gram-schmidt/Content licensed under CC BY-NC 4.0.