Panorama Stitching

Every time you sweep your phone in a slow arc and tap the shutter, a small algorithm quietly performs a geometric miracle: it aligns dozens of overlapping frames, warps each one onto a shared canvas, and hides the stitching line so well that the result looks like a single photograph. This is panorama stitching — and the mathematics underneath it is the same projective geometry that powers 3-D reconstruction, augmented reality, and satellite mapping.

The core insight is surprisingly elegant: two photos of the same flat surface (or of a scene photographed from a single rotating camera) are related by a homography — a $3 \times 3$ matrix that maps every pixel from one image onto the other. Once you have that matrix you can warp one photo until it lines up perfectly with its neighbor, then find a seam that crosses the least-visible boundary and blend both sides together.

Three algorithmic steps make it happen:

1. Feature detection and matching — find distinctive keypoints (corners, blobs) in both images and pair up the ones that look alike.
2. Homography estimation — use those point pairs to solve for the transformation matrix, discarding bad matches with RANSAC.
3. Warping and blending — project one image onto the other's coordinate system, then cut a seam that avoids high-contrast edges and fade both sides across it.

The result lands in your camera roll in under a second. Understanding each step reveals a beautiful interplay of linear algebra, robust statistics, and combinatorial optimization.

The demo below simulates the core pipeline: two synthetic image strips share an overlapping region. Adjust the overlap slider, then click Stitch to watch the algorithm warp the right strip onto the left one and find the least-visible seam.

<!-- {{c_intro}} -->
<p class="hint">{{hint_para}}</p>
<div class="controls">
  <label for="overlap">{{lbl_overlap}} <span id="overlap-val">40</span>%</label>
  <input type="range" id="overlap" min="15" max="65" value="40" step="5">
  <button id="btn-stitch" type="button">{{btn_stitch}}</button>
  <button id="btn-reset" type="button" class="ghost">{{btn_reset}}</button>
</div>
<div class="canvas-wrap">
  <canvas id="cvs" width="600" height="180"></canvas>
</div>
<div class="status" id="status"></div>
<div class="legend">
  <span class="swatch left-swatch"></span>{{legend_left}}
  <span class="swatch right-swatch"></span>{{legend_right}}
  <span class="swatch seam-swatch"></span>{{legend_seam}}
</div>

/* {{c_styles}} */
* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; color: #222; margin: 0; padding: .5rem; }
.hint { font-size: .88rem; color: #444; margin: 0 0 .6rem; line-height: 1.45; }
.controls { display: flex; align-items: center; gap: .6rem; flex-wrap: wrap; margin-bottom: .7rem; }
label { font-size: .88rem; white-space: nowrap; }
input[type=range] { width: 120px; cursor: pointer; }
button { font: 600 14px system-ui; padding: .4rem .85rem; border: 1px solid #1d3557;
         background: #1d3557; color: #fff; border-radius: 8px; cursor: pointer; }
button.ghost { background: #fff; color: #1d3557; }
.canvas-wrap { border: 1px solid #cdd9e3; border-radius: 8px; overflow: hidden; display: inline-block; }
canvas { display: block; }
.status { font-size: .95rem; font-weight: 600; min-height: 1.4em; margin: .4rem 0; }
.status.ok { color: #0a7d33; }
.status.info { color: #1d3557; }
.legend { display: flex; align-items: center; gap: .5rem; font-size: .82rem; flex-wrap: wrap; margin-top: .3rem; }
.swatch { display: inline-block; width: 14px; height: 14px; border-radius: 3px; border: 1px solid #aaa; }
.left-swatch { background: #4fa3d1; }
.right-swatch { background: #e87d4f; }
.seam-swatch { background: #f5e642; border-color: #bba800; }

// Code not found

Notice how the seam always falls where the two images agree most — across the uniform region, never across a sharp edge. Move the overlap to the minimum and the blending must work harder; push it to the maximum and there is plenty of shared content to anchor the warp precisely.

Each stage of the pipeline has its own complexity story.

Homography estimation. A homography has 8 degrees of freedom (the $3 \times 3$ matrix up to scale). Each matched point pair gives 2 equations, so 4 pairs suffice in principle. With $n$ matched pairs the linear system is overdetermined and solved by least squares in $O(n)$ time. The hard part is which pairs to trust.

RANSAC (Random Sample Consensus). Most feature matches are wrong ("outliers"). RANSAC draws a random minimal sample of 4 pairs, estimates a homography, counts how many of the remaining matches agree with it (the "inliers"), and repeats. After $k$ rounds the probability that at least one round drew 4 inliers is $1 - (1 - p^4)^k$, where $p$ is the fraction of true inliers. With 50 % inliers, just $k = 17$ rounds give a 99 % success rate. This is why RANSAC is so effective: the exponential that makes brute-force search hopeless is now in your favor.

Seam finding. After warping, the overlap zone has two candidate pixels at every location. A naive search for the best-looking cut is $O(2^{\text{width}})$ — impossible. Dynamic programming collapses it to $O(w \cdot h)$: scan top-to-bottom, and at each pixel keep only the cheapest path from the top edge. This is essentially the same recurrence as sequence alignment in computational biology, applied to image gradients instead of DNA letters.

Multi-band blending (Laplacian pyramids) averages low-frequency color differences over wide bands and high-frequency detail over narrow bands, eliminating color seams without blurring edges — at the cost of $O(n \log n)$ pyramid construction.

Together the pipeline is fast enough to run on a phone in real time, yet grounded in the same ideas — least squares, probabilistic sampling, dynamic programming — that appear throughout algorithms and complexity.

The same three-step pipeline — detect, warp, blend — appears across a surprising range of fields:

• Satellite and aerial mapping: agencies like NASA and ESA stitch thousands of overlapping swaths into seamless global mosaics. The homography generalizes to a rational polynomial camera model, but RANSAC and seam finding stay the same.
• Medical imaging: whole-slide scanners stitch microscope frames into gigapixel images of tissue samples; surgical robots stitch endoscope frames to give surgeons a wider field of view without moving the scope.
• Video stabilization: aligning consecutive frames with a homography removes camera shake; the warp that undoes the motion is exactly the inverse of the estimated transformation.
• Augmented reality: overlaying a virtual object on a real surface requires knowing the homography between the camera's view and the surface plane — the same matrix computed in stitching, now used in reverse to place pixels precisely.
• Document scanning: phone-based document scanners warp a photographed page onto a rectangle by estimating the homography from its four corners — a four-point version of the same estimation step.

Understand panorama stitching and you hold the key to all projective-geometry applications in computer vision.

Panorama stitching is a study in algorithmic layers: each step solves a genuinely hard subproblem with a technique that is efficient precisely because it exploits the structure of the data.

Feature matching finds the correspondence; RANSAC filters the noise without exhaustive search; homography estimation solves a linear system in milliseconds; dynamic programming finds the optimal seam in linear time; and Laplacian blending hides the join without blurring the details. Stack those five ingredients and a single swipe of your phone produces a seamless wide-angle photograph.

The deeper lesson is one of reduction: a problem that looks dauntingly complex — align and merge two images so the join is invisible — turns out to decompose into well-understood pieces, each solvable with tools you can find in any algorithms textbook. That is the quiet power of algorithmic thinking, and it is why the same ideas keep appearing from your camera roll to the control room of a satellite.