Image Pyramids

Look at a photograph from across a room, then walk up to it. You notice different things at each distance: the overall composition from afar, the texture of brushstrokes up close. Image pyramids give a computer that same ability — to see a picture at many scales simultaneously.

The idea is elegantly simple. Start with the original image. Blur it slightly (a Gaussian filter), then shrink it by half. Blur and shrink again. Repeat until you have a thumbnail. Stack those copies from big to small, like a pyramid. Each level captures structure at a coarser scale; fine detail lives at the bottom, global shape at the top.

Peter Burt and Edward Adelson formalised this in 1983 with two complementary structures:

• The Gaussian pyramid is that stack of progressively blurred, down-sampled copies. Each level is a low-pass filtered version of the level below.
• The Laplacian pyramid captures what the Gaussian pyramid throws away. Each level stores the difference between a level and the up-sampled version of the level above — a band-pass residual. Summing all Laplacian levels perfectly reconstructs the original image.

These two structures are the backbone of a surprising range of algorithms — from the seamless photo blending you will try in the demo, to the feature detectors inside modern compression pipelines and object-recognition systems.

The classic demonstration of Laplacian pyramids is seamless image blending: merge the left half of one image with the right half of another without a visible seam. A simple hard cut looks jarring; pyramid blending looks natural.

<!-- {{c_html_intro}} -->
<div class="controls">
  <label for="levels">{{label_levels}} <strong id="lvlVal">4</strong></label>
  <input id="levels" type="range" min="1" max="6" value="4" step="1">
  <button id="btnBlend" type="button">{{btn_blend}}</button>
  <button id="btnReset" type="button" class="ghost">{{btn_reset}}</button>
</div>
<div class="canvases">
  <div class="panel">
    <div class="panel-label">{{label_img_a}}</div>
    <canvas id="cA" width="128" height="128"></canvas>
  </div>
  <div class="panel">
    <div class="panel-label">{{label_img_b}}</div>
    <canvas id="cB" width="128" height="128"></canvas>
  </div>
  <div class="panel">
    <div class="panel-label">{{label_result}}</div>
    <canvas id="cOut" width="128" height="128"></canvas>
  </div>
</div>
<div id="status" class="status">{{status_ready}}</div>
<div class="note">{{note_text}}</div>

/* {{c_css_intro}} */
* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; color: #222; margin: 0; }
.controls { display: flex; flex-wrap: wrap; align-items: center; gap: .5rem; margin-bottom: .8rem; }
label { font-size: .9rem; }
input[type=range] { width: 120px; }
button { font: 600 14px system-ui; padding: .4rem .85rem; border: 1px solid #1d3557;
         background: #1d3557; color: #fff; border-radius: 8px; cursor: pointer; }
button.ghost { background: #fff; color: #1d3557; }
.canvases { display: flex; flex-wrap: wrap; gap: 1rem; margin: .5rem 0; }
.panel { display: flex; flex-direction: column; align-items: center; gap: .3rem; }
.panel-label { font-size: .8rem; font-weight: 600; color: #555; text-transform: uppercase; letter-spacing: .04em; }
canvas { border: 1px solid #ccc; border-radius: 4px; image-rendering: pixelated; width: 128px; height: 128px; }
.status { font-size: .95rem; font-weight: 600; min-height: 1.4em; margin: .3rem 0; }
.status.ok { color: #0a7d33; }
.note { font-size: .82rem; color: #555; line-height: 1.45; max-width: 420px; }

// Code not found

Use the slider to choose how many pyramid levels participate in the blend. With just one level you are back to a hard cut — the seam is obvious. As you add levels, low-frequency (large-scale) structure blends over a wide transition zone while high-frequency (fine) detail still blends sharply near the seam. The result looks as if the two images always belonged together.

This is the key insight: different frequencies need different blend widths. A pyramid separates an image into frequency bands, blends each band at its natural scale, then reconstructs — exactly what the Laplacian pyramid was built for.

Let $G_0$ be the original image. The Gaussian pyramid is the sequence $G_0, G_1, G_2, \dots$ where each level is obtained by convolving with a Gaussian kernel $h$ and then dropping every other row and column:

$$G_{k+1}[i,j] = \sum_{m,n} h[m,n]\, G_k[2i+m,\, 2j+n]$$

This is the reduce operation — blur, then sub-sample. The blurring prevents aliasing: high frequencies that would fold back onto lower ones are killed first.

The Laplacian pyramid level $L_k$ is the difference between $G_k$ and the expand of $G_{k+1}$:

$$L_k = G_k - \text{expand}(G_{k+1})$$

where expand$(G_{k+1})$ inserts zeros between samples and re-filters. $L_k$ is a band-pass image — it holds only the frequencies present at scale $k$ but absent at scale $k+1$.

Perfect reconstruction follows by induction: start from the coarsest Gaussian level $G_N$ and apply:

$$G_k = L_k + \text{expand}(G_{k+1})$$

climbing back down the pyramid until you recover $G_0$ exactly (up to rounding in fixed-point arithmetic).

For blending, given images $A$ and $B$ and a soft mask $M$ (values 0–1), the algorithm builds $L^A_k$, $L^B_k$, and $G^M_k$ (the Gaussian pyramid of $M$), then blends each Laplacian level:

$$L^C_k = G^M_k \cdot L^A_k + (1 - G^M_k) \cdot L^B_k$$

Reconstructing from $L^C$ gives a blend where each frequency component transitions at its own natural scale — no visible seam, no halo artifacts. The same formalism connects to compression: JPEG 2000's wavelet transform is a close cousin, and so is the scale-space used by the SIFT feature detector.

Image pyramids are not a curiosity — they are inside most software that touches images:

• Photo blending and panorama stitching: every modern phone's "Panorama" mode uses pyramid blending to hide the seams between overlapping frames. The same technique underlies HDR exposure fusion.
• Object detection: the original Viola–Jones face detector scans a fixed-size filter over every level of the Gaussian pyramid to find faces at different distances. Modern neural detectors use learned feature pyramid networks (FPNs) that extend the same idea.
• Image compression: JPEG 2000 and many video codecs decompose frames into frequency subbands — essentially a Laplacian pyramid — and quantise each band separately. Coarse bands get fewer bits; fine-detail bands are discarded first when bandwidth is tight.
• Optical flow: algorithms like Lucas–Kanade estimate motion starting at the coarsest pyramid level (where large displacements look small) and refine the estimate level by level.
• Texture synthesis: multi-resolution techniques match texture statistics across pyramid levels to produce seamless, large-scale synthetic textures.

The unifying thread is the same as in the compression article: separating a signal into scale-specific components lets each component be processed optimally, whether the goal is blending, detecting, or coding.

Burt and Adelson's 1983 insight was beautifully simple: represent an image not as pixels but as a stack of differences across scales. The Gaussian pyramid captures what is visible at each resolution; the Laplacian pyramid captures what each resolution adds that the coarser one missed. Together they form a lossless, invertible decomposition.

That decomposition turns out to be the right tool for a surprisingly large slice of computer vision. Blending without seams, detecting objects at any size, compressing video efficiently, tracking motion across large displacements — all reduce to processing each frequency band at its natural scale.

The next time a photo app stitches your panorama seamlessly, or a streaming service delivers crisp video on a slow connection, a pyramid is almost certainly doing the heavy lifting — one blurred, shrunk layer at a time.