Introduction

Every time you sweep your phone in a slow arc and tap the shutter, a small algorithm quietly performs a geometric miracle: it aligns dozens of overlapping frames, warps each one onto a shared canvas, and hides the stitching line so well that the result looks like a single photograph. This is panorama stitching — and the mathematics underneath it is the same projective geometry that powers 3-D reconstruction, augmented reality, and satellite mapping.

The core insight is surprisingly elegant: two photos of the same flat surface (or of a scene photographed from a single rotating camera) are related by a homography — a 3×33 \times 3 matrix that maps every pixel from one image onto the other. Once you have that matrix you can warp one photo until it lines up perfectly with its neighbor, then find a seam that crosses the least-visible boundary and blend both sides together.

Three algorithmic steps make it happen:

  1. Feature detection and matching — find distinctive keypoints (corners, blobs) in both images and pair up the ones that look alike.
  2. Homography estimation — use those point pairs to solve for the transformation matrix, discarding bad matches with RANSAC.
  3. Warping and blending — project one image onto the other's coordinate system, then cut a seam that avoids high-contrast edges and fade both sides across it.

The result lands in your camera roll in under a second. Understanding each step reveals a beautiful interplay of linear algebra, robust statistics, and combinatorial optimization.

Try It

The demo below simulates the core pipeline: two synthetic image strips share an overlapping region. Adjust the overlap slider, then click Stitch to watch the algorithm warp the right strip onto the left one and find the least-visible seam.

<!-- Panorama stitching demo: warp + seam-cut + blend two synthetic image strips. -->
<p class="hint">Two synthetic image strips overlap. Adjust the overlap width, then click <b>Stitch</b> to estimate the warp, find the optimal seam via dynamic programming, and blend the result. The yellow line shows where the seam falls.</p>
<div class="controls">
  <label for="overlap">Overlap: <span id="overlap-val">40</span>%</label>
  <input type="range" id="overlap" min="15" max="65" value="40" step="5">
  <button id="btn-stitch" type="button">Stitch</button>
  <button id="btn-reset" type="button" class="ghost">Reset</button>
</div>
<div class="canvas-wrap">
  <canvas id="cvs" width="600" height="180"></canvas>
</div>
<div class="status" id="status"></div>
<div class="legend">
  <span class="swatch left-swatch"></span>Left strip
  <span class="swatch right-swatch"></span>Right strip
  <span class="swatch seam-swatch"></span>Seam
</div>

/* Styles for the panorama stitching demo */
* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; color: #222; margin: 0; padding: .5rem; }
.hint { font-size: .88rem; color: #444; margin: 0 0 .6rem; line-height: 1.45; }
.controls { display: flex; align-items: center; gap: .6rem; flex-wrap: wrap; margin-bottom: .7rem; }
label { font-size: .88rem; white-space: nowrap; }
input[type=range] { width: 120px; cursor: pointer; }
button { font: 600 14px system-ui; padding: .4rem .85rem; border: 1px solid #1d3557;
         background: #1d3557; color: #fff; border-radius: 8px; cursor: pointer; }
button.ghost { background: #fff; color: #1d3557; }
.canvas-wrap { border: 1px solid #cdd9e3; border-radius: 8px; overflow: hidden; display: inline-block; }
canvas { display: block; }
.status { font-size: .95rem; font-weight: 600; min-height: 1.4em; margin: .4rem 0; }
.status.ok { color: #0a7d33; }
.status.info { color: #1d3557; }
.legend { display: flex; align-items: center; gap: .5rem; font-size: .82rem; flex-wrap: wrap; margin-top: .3rem; }
.swatch { display: inline-block; width: 14px; height: 14px; border-radius: 3px; border: 1px solid #aaa; }
.left-swatch { background: #4fa3d1; }
.right-swatch { background: #e87d4f; }
.seam-swatch { background: #f5e642; border-color: #bba800; }

const i18nFmt = (s, v) => s.replace(/%(\w+)%/g, (_, k) => String(v[k] ?? ''));
// Canvas and context for the panorama demo
const cvs = document.getElementById('cvs');
const ctx = cvs.getContext('2d');
const W = 600, H = 180;

// Synthetic image strips — left is blue-ish, right is orange-ish
// Left strip: blue-ish gradient with a vertical dark stripe near the center
// Right strip: orange-ish gradient with a vertical bright stripe near the center
// They share an overlapping region whose width is controlled by the slider.

function makeStrip(side) {
  // Build a synthetic image strip pixel by pixel
  const img = ctx.createImageData(W, H);
  for (let y = 0; y < H; y++) {
    for (let x = 0; x < W; x++) {
      const t = x / W;
      let r, g, b;
      if (side === 'left') {
        // Blue-ish base gradient for the left strip
        r = Math.round(30 + t * 80);
        g = Math.round(100 + t * 60);
        b = Math.round(180 + t * 40);
        // Dark vertical stripe for visual interest in the left strip
        if (x > W * 0.38 && x < W * 0.45) { r = Math.max(0, r - 60); g = Math.max(0, g - 60); b = Math.max(0, b - 60); }
      } else {
        // Orange-ish base gradient for the right strip
        r = Math.round(200 + t * 30);
        g = Math.round(120 - t * 40);
        b = Math.round(60 + t * 20);
        // Bright vertical stripe for visual interest in the right strip
        if (x > W * 0.35 && x < W * 0.42) { r = Math.min(255, r + 50); g = Math.min(255, g + 50); b = Math.min(255, b + 50); }
      }
      // Add a sinusoidal wave texture to both strips
      const wave = Math.round(12 * Math.sin((y / H) * Math.PI * 4 + (side === 'left' ? 0 : 1)));
      r = Math.max(0, Math.min(255, r + wave));
      g = Math.max(0, Math.min(255, g + wave));
      b = Math.max(0, Math.min(255, b + wave));
      const i = (y * W + x) * 4;
      img.data[i] = r; img.data[i+1] = g; img.data[i+2] = b; img.data[i+3] = 255;
    }
  }
  return img;
}

// Homography (simplified to translation) that aligns the right strip onto the panorama canvas
// The "homography" here is a simple horizontal translation that maps
// the right strip so its left edge aligns with the overlap boundary
// in the left strip. A real homography is a full projective transform;
// for strips photographed with a pan this reduces to a translation + scale.
function warpRight(rightImg, overlapPx, outW) {
  // Warp the right strip by shifting it left by (W - overlapPx) pixels
  const out = ctx.createImageData(outW, H);
  const shift = W - overlapPx;
  for (let y = 0; y < H; y++) {
    for (let x = 0; x < outW; x++) {
      const srcX = x - shift;
      if (srcX < 0 || srcX >= W) continue;
      const si = (y * W + srcX) * 4;
      const di = (y * outW + x) * 4;
      out.data[di] = rightImg.data[si];
      out.data[di+1] = rightImg.data[si+1];
      out.data[di+2] = rightImg.data[si+2];
      out.data[di+3] = rightImg.data[si+3];
    }
  }
  return out;
}

// Seam finding using dynamic programming on the overlap band
// Seam finding: dynamic programming on the overlap band.
// Cost at each pixel = |left_brightness - right_brightness|.
// We find the minimum-cost vertical path through the overlap zone
// (each step can go straight, diagonal-left or diagonal-right).
function findSeam(leftImg, warpedRight, seamStart, seamEnd) {
  // Build cost matrix — cost = absolute brightness difference between left and right
  const cost = [];
  for (let y = 0; y < H; y++) {
    cost.push([]);
    for (let x = seamStart; x <= seamEnd; x++) {
      const li = (y * W + x) * 4;
      const ri = (y * W + x) * 4;
      const lBright = (leftImg.data[li] + leftImg.data[li+1] + leftImg.data[li+2]) / 3;
      const rBright = (warpedRight.data[ri] + warpedRight.data[ri+1] + warpedRight.data[ri+2]) / 3;
      cost[y].push(Math.abs(lBright - rBright));
    }
  }
  const cols = seamEnd - seamStart + 1;
  // Fill DP table row by row, tracking the minimum-cost predecessor
  const dp = Array.from({ length: H }, () => new Float32Array(cols).fill(Infinity));
  const from = Array.from({ length: H }, () => new Int16Array(cols).fill(-1));
  for (let c = 0; c < cols; c++) dp[0][c] = cost[0][c];
  for (let y = 1; y < H; y++) {
    for (let c = 0; c < cols; c++) {
      let best = dp[y-1][c], bestC = c;
      if (c > 0 && dp[y-1][c-1] < best) { best = dp[y-1][c-1]; bestC = c-1; }
      if (c < cols-1 && dp[y-1][c+1] < best) { best = dp[y-1][c+1]; bestC = c+1; }
      dp[y][c] = cost[y][c] + best;
      from[y][c] = bestC;
    }
  }
  // Backtrack through the DP table to recover the optimal seam path
  const seam = new Int32Array(H);
  let cur = 0;
  for (let c = 1; c < cols; c++) if (dp[H-1][c] < dp[H-1][cur]) cur = c;
  seam[H-1] = seamStart + cur;
  for (let y = H-2; y >= 0; y--) { cur = from[y+1][cur]; seam[y] = seamStart + cur; }
  return seam;
}

// Blending — left of seam comes from left strip, right comes from warped right strip, with a feather at the seam
// Blending: left of the seam = left image, right of seam = warped right image.
// A narrow feather (5 px) is applied at the seam for a smoother join.
function blend(leftImg, warpedRight, seam, outW) {
  // Blend pixel by pixel using the seam as the boundary with a narrow feather region
  const out = ctx.createImageData(outW, H);
  const FEATHER = 5;
  for (let y = 0; y < H; y++) {
    const sx = seam[y];
    for (let x = 0; x < outW; x++) {
      let alpha;
      if (x < sx - FEATHER) {
        alpha = 1;
      } else if (x > sx + FEATHER) {
        alpha = 0;
      } else {
        alpha = (sx + FEATHER - x) / (2 * FEATHER);
      }
      const hasR = warpedRight.data[(y * outW + x) * 4 + 3] > 0;
      const hasL = x < W;
      let r, g, b;
      const li = hasL ? (y * W + x) * 4 : -1;
      const ri = (y * outW + x) * 4;
      if (!hasR || (hasL && alpha >= 1)) {
        r = hasL ? leftImg.data[li] : 0;
        g = hasL ? leftImg.data[li+1] : 0;
        b = hasL ? leftImg.data[li+2] : 0;
      } else if (!hasL || alpha <= 0) {
        r = warpedRight.data[ri]; g = warpedRight.data[ri+1]; b = warpedRight.data[ri+2];
      } else {
        r = Math.round(alpha * leftImg.data[li] + (1-alpha) * warpedRight.data[ri]);
        g = Math.round(alpha * leftImg.data[li+1] + (1-alpha) * warpedRight.data[ri+1]);
        b = Math.round(alpha * leftImg.data[li+2] + (1-alpha) * warpedRight.data[ri+2]);
      }
      const di = (y * outW + x) * 4;
      out.data[di] = r; out.data[di+1] = g; out.data[di+2] = b; out.data[di+3] = 255;
    }
  }
  return out;
}

// Draw left and right strips side by side for the initial view
function drawSideBySide(leftImg, rightImg) {
  const half = W / 2;
  const tmp = document.createElement('canvas');
  tmp.width = W; tmp.height = H;
  const tc = tmp.getContext('2d');
  tc.putImageData(leftImg, 0, 0);
  ctx.drawImage(tmp, 0, 0, W, H, 0, 0, half, H);
  tc.putImageData(rightImg, 0, 0);
  ctx.drawImage(tmp, 0, 0, W, H, half, 0, half, H);
  // Draw a white divider between the two strips
  ctx.strokeStyle = '#fff';
  ctx.lineWidth = 2;
  ctx.beginPath(); ctx.moveTo(half, 0); ctx.lineTo(half, H); ctx.stroke();
}

let leftImg = null, rightImg = null;
let stitched = false;

function init() {
  // Initialize both synthetic strips and draw side by side
  leftImg = makeStrip('left');
  rightImg = makeStrip('right');
  drawSideBySide(leftImg, rightImg);
  stitched = false;
  setStatus('Adjust overlap and click Stitch to run the pipeline.', 'info');
}

function setStatus(msg, cls) {
  const el = document.getElementById('status');
  el.textContent = msg;
  el.className = 'status ' + cls;
}

function stitch() {
  // Run the full stitch pipeline — warp, seam, blend
  const overlapPct = parseInt(document.getElementById('overlap').value, 10);
  const overlapPx = Math.round(W * overlapPct / 100);
  const outW = 2 * W - overlapPx;
  // Step 1 — warp the right strip onto the panorama canvas
  const warped = warpRight(rightImg, overlapPx, outW);
  // Step 2 — find the optimal seam in the overlap region using DP
  const seamStart = W - overlapPx;
  const seamEnd = W - 1;
  const leftPadded = ctx.createImageData(outW, H);
  for (let y = 0; y < H; y++) {
    for (let x = 0; x < W; x++) {
      const si = (y * W + x) * 4;
      const di = (y * outW + x) * 4;
      leftPadded.data[di] = leftImg.data[si];
      leftPadded.data[di+1] = leftImg.data[si+1];
      leftPadded.data[di+2] = leftImg.data[si+2];
      leftPadded.data[di+3] = leftImg.data[si+3];
    }
  }
  const seam = findSeam(leftPadded, warped, seamStart, seamEnd);
  // Step 3 — blend left and right across the seam
  const blended = blend(leftPadded, warped, seam, outW);
  // Scale the panorama to fit the canvas width
  const tmp = document.createElement('canvas');
  tmp.width = outW; tmp.height = H;
  const tc = tmp.getContext('2d');
  tc.putImageData(blended, 0, 0);
  ctx.clearRect(0, 0, W, H);
  ctx.drawImage(tmp, 0, 0, outW, H, 0, 0, W, H);
  // Overlay the seam path in yellow
  ctx.strokeStyle = '#f5e642';
  ctx.lineWidth = 1.5;
  ctx.beginPath();
  for (let y = 0; y < H; y++) {
    const seamX = Math.round(seam[y] * W / outW);
    if (y === 0) ctx.moveTo(seamX, y);
    else ctx.lineTo(seamX, y);
  }
  ctx.stroke();
  stitched = true;
  setStatus(i18nFmt('Stitched! Overlap: %overlap%% — seam follows the least-contrast path.', { overlap: overlapPct }), 'ok');
}

document.getElementById('overlap').addEventListener('input', function() {
  document.getElementById('overlap-val').textContent = this.value;
  if (stitched) init();
});
document.getElementById('btn-stitch').addEventListener('click', stitch);
document.getElementById('btn-reset').addEventListener('click', init);
init();
Result
Console

Notice how the seam always falls where the two images agree most — across the uniform region, never across a sharp edge. Move the overlap to the minimum and the blending must work harder; push it to the maximum and there is plenty of shared content to anchor the warp precisely.

The Real Complexity

Each stage of the pipeline has its own complexity story.

Homography estimation. A homography has 8 degrees of freedom (the 3×33 \times 3 matrix up to scale). Each matched point pair gives 2 equations, so 4 pairs suffice in principle. With nn matched pairs the linear system is overdetermined and solved by least squares in O(n)O(n) time. The hard part is which pairs to trust.

RANSAC (Random Sample Consensus). Most feature matches are wrong ("outliers"). RANSAC draws a random minimal sample of 4 pairs, estimates a homography, counts how many of the remaining matches agree with it (the "inliers"), and repeats. After kk rounds the probability that at least one round drew 4 inliers is 1−(1−p4)k1 - (1 - p^4)^k, where pp is the fraction of true inliers. With 50 % inliers, just k=17k = 17 rounds give a 99 % success rate. This is why RANSAC is so effective: the exponential that makes brute-force search hopeless is now in your favor.

Seam finding. After warping, the overlap zone has two candidate pixels at every location. A naive search for the best-looking cut is O(2width)O(2^{\text{width}}) — impossible. Dynamic programming collapses it to O(w⋅h)O(w \cdot h): scan top-to-bottom, and at each pixel keep only the cheapest path from the top edge. This is essentially the same recurrence as sequence alignment in computational biology, applied to image gradients instead of DNA letters.

Multi-band blending (Laplacian pyramids) averages low-frequency color differences over wide bands and high-frequency detail over narrow bands, eliminating color seams without blurring edges — at the cost of O(nlog⁡n)O(n \log n) pyramid construction.

Together the pipeline is fast enough to run on a phone in real time, yet grounded in the same ideas — least squares, probabilistic sampling, dynamic programming — that appear throughout algorithms and complexity.

Where It Matters

The same three-step pipeline — detect, warp, blend — appears across a surprising range of fields:

  • Satellite and aerial mapping: agencies like NASA and ESA stitch thousands of overlapping swaths into seamless global mosaics. The homography generalizes to a rational polynomial camera model, but RANSAC and seam finding stay the same.
  • Medical imaging: whole-slide scanners stitch microscope frames into gigapixel images of tissue samples; surgical robots stitch endoscope frames to give surgeons a wider field of view without moving the scope.
  • Video stabilization: aligning consecutive frames with a homography removes camera shake; the warp that undoes the motion is exactly the inverse of the estimated transformation.
  • Augmented reality: overlaying a virtual object on a real surface requires knowing the homography between the camera's view and the surface plane — the same matrix computed in stitching, now used in reverse to place pixels precisely.
  • Document scanning: phone-based document scanners warp a photographed page onto a rectangle by estimating the homography from its four corners — a four-point version of the same estimation step.

Understand panorama stitching and you hold the key to all projective-geometry applications in computer vision.

Conclusion

Panorama stitching is a study in algorithmic layers: each step solves a genuinely hard subproblem with a technique that is efficient precisely because it exploits the structure of the data.

Feature matching finds the correspondence; RANSAC filters the noise without exhaustive search; homography estimation solves a linear system in milliseconds; dynamic programming finds the optimal seam in linear time; and Laplacian blending hides the join without blurring the details. Stack those five ingredients and a single swipe of your phone produces a seamless wide-angle photograph.

The deeper lesson is one of reduction: a problem that looks dauntingly complex — align and merge two images so the join is invisible — turns out to decompose into well-understood pieces, each solvable with tools you can find in any algorithms textbook. That is the quiet power of algorithmic thinking, and it is why the same ideas keep appearing from your camera roll to the control room of a satellite.

Share this article

Pick a channel — or use your device's native share sheet.

Comments

Loading comments...

https://www.kipuhub.com/en/article/seam-stitching-panorama/Content licensed under CC BY-NC 4.0.