Harris Corner Detection

Introduction

Look at any photograph and pick a point at random. If it lies on a smooth wall, the color barely changes in any direction — it carries almost no information. If it lies on a straight edge, intensity changes sharply across the edge but not along it, so you can slide along the edge without knowing where you are. But if it sits at a corner — where two edges meet — intensity changes strongly in every direction. You are pinned.

That simple insight drives Harris Corner Detection, introduced by Chris Harris and Mike Stephens in 1988. By measuring how much brightness changes when you shift a small window in any direction, the algorithm assigns every pixel a corner response score. High score means corner; near-zero means flat; a score that is high in one direction but low in another means edge.

The result is a set of stable, repeatable landmark points — the same physical corner lights up whenever the camera sees it, regardless of small shifts in viewpoint or lighting. Those landmarks are the foundation of almost everything modern computer vision does: stitching panoramas, tracking objects, reconstructing 3-D scenes from video, and matching features across images taken from completely different angles.

Try It

The canvas below draws a checkerboard pattern and runs the Harris detector on every pixel. Red dots mark the corners that scored above the threshold; the color of each pixel's background shows its raw corner response (brighter = stronger response).

<!-- Harris Corner Detection demo — checkerboard with overlay -->
<div class="controls">
  <label>Threshold: <span id="threshVal">40</span>%
    <input type="range" id="thresh" min="1" max="99" value="40">
  </label>
  <label>Cell size:
    <select id="cellSz">
      <option value="16">16 px</option>
      <option value="24" selected>24 px</option>
      <option value="32">32 px</option>
    </select>
  </label>
  <button id="btnRun" type="button">Redraw</button>
</div>
<div class="canvas-wrap">
  <canvas id="cvs"></canvas>
</div>
<div class="legend">
  <span class="dot corner"></span> Corner
  <span class="dot edge"></span> Edge / gradient
  <span class="dot flat"></span> Flat
</div>
<div id="status" class="status"></div>

/* Styles for the Harris corner detection demo */
* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; margin: 0; color: #222; }
.controls { display: flex; gap: .7rem; flex-wrap: wrap; align-items: center; margin-bottom: .6rem; }
label { font-size: .88rem; display: flex; align-items: center; gap: .35rem; }
input[type=range] { width: 110px; cursor: pointer; }
select { font-size: .88rem; border: 1px solid #aaa; border-radius: 6px; padding: .15rem .3rem; }
button { font: 600 13px system-ui; padding: .4rem .85rem; border: 1px solid #1d3557;
         background: #1d3557; color: #fff; border-radius: 8px; cursor: pointer; }
.canvas-wrap { overflow: auto; max-width: 100%; }
canvas { display: block; image-rendering: pixelated; border: 1px solid #ccc; border-radius: 4px; }
.legend { font-size: .82rem; margin-top: .5rem; display: flex; gap: .9rem; align-items: center; }
.dot { display: inline-block; width: 12px; height: 12px; border-radius: 50%; }
.dot.corner { background: #e63946; }
.dot.edge    { background: #457b9d; }
.dot.flat    { background: #ccc; }
.status { font-size: .88rem; font-weight: 600; margin-top: .4rem; min-height: 1.2em; color: #1d3557; }

/* Harris Corner Detection demo — structure tensor on a checkerboard */
const i18nFmt = (s, v) => s.replace(/%(\w+)%/g, (_, k) => String(v[k] ?? ''));

const cvs    = document.getElementById('cvs');
const ctx    = cvs.getContext('2d');
const thresh = document.getElementById('thresh');
const threshVal = document.getElementById('threshVal');
const cellSzSel = document.getElementById('cellSz');
const statusEl  = document.getElementById('status');

/* Draw a black-and-white checkerboard on the canvas */
function drawCheckerboard(cellPx, cols, rows) {
  cvs.width  = cols * cellPx;
  cvs.height = rows * cellPx;
  for (let r = 0; r < rows; r++) {
    for (let c = 0; c < cols; c++) {
      ctx.fillStyle = (r + c) % 2 === 0 ? '#ffffff' : '#222222';
      ctx.fillRect(c * cellPx, r * cellPx, cellPx, cellPx);
    }
  }
}

/* Compute Sobel gradients Ix and Iy for every pixel */
function sobelGradients(img, W, H) {
  const d = img.data;
  const Ix = new Float32Array(W * H);
  const Iy = new Float32Array(W * H);
  const gray = new Float32Array(W * H);

  /* Convert to grayscale using luminance weights */
  for (let i = 0; i < W * H; i++) {
    gray[i] = 0.299 * d[i * 4] + 0.587 * d[i * 4 + 1] + 0.114 * d[i * 4 + 2];
  }

  /* Sobel 3x3 kernel for horizontal and vertical gradients */
  for (let y = 1; y < H - 1; y++) {
    for (let x = 1; x < W - 1; x++) {
      const idx = y * W + x;
      Ix[idx] =
        -gray[(y-1)*W+(x-1)] + gray[(y-1)*W+(x+1)]
        - 2*gray[y*W+(x-1)]  + 2*gray[y*W+(x+1)]
        - gray[(y+1)*W+(x-1)] + gray[(y+1)*W+(x+1)];
      Iy[idx] =
        -gray[(y-1)*W+(x-1)] - 2*gray[(y-1)*W+x] - gray[(y-1)*W+(x+1)]
        + gray[(y+1)*W+(x-1)] + 2*gray[(y+1)*W+x] + gray[(y+1)*W+(x+1)];
    }
  }
  return { Ix, Iy };
}

/* Compute Harris corner response R = det(M) - k * trace(M)^2 for each pixel */
function harrisResponse(Ix, Iy, W, H, winR, k) {
  const R = new Float32Array(W * H);
  for (let y = winR; y < H - winR; y++) {
    for (let x = winR; x < W - winR; x++) {
      let Sxx = 0, Sxy = 0, Syy = 0;
      /* Accumulate the 2x2 structure tensor over the window */
      for (let dy = -winR; dy <= winR; dy++) {
        for (let dx = -winR; dx <= winR; dx++) {
          const i = (y + dy) * W + (x + dx);
          Sxx += Ix[i] * Ix[i];
          Sxy += Ix[i] * Iy[i];
          Syy += Iy[i] * Iy[i];
        }
      }
      /* R = det - k * trace^2 (Harris 1988) */
      const det   = Sxx * Syy - Sxy * Sxy;
      const trace = Sxx + Syy;
      R[y * W + x] = det - k * trace * trace;
    }
  }
  return R;
}

/* Main run function — draw checkerboard then compute and overlay Harris response */
function run() {
  const cellPx = parseInt(cellSzSel.value, 10);
  const cols = 10, rows = 10;

  drawCheckerboard(cellPx, cols, rows);
  const W = cvs.width, H = cvs.height;
  const img = ctx.getImageData(0, 0, W, H);

  const { Ix, Iy } = sobelGradients(img, W, H);
  const R = harrisResponse(Ix, Iy, W, H, 2, 0.04);

  /* Find the max positive response for normalization */
  let maxR = 0;
  for (let i = 0; i < R.length; i++) if (R[i] > maxR) maxR = R[i];

  /* Absolute threshold from the slider percentage */
  const threshFrac = parseInt(thresh.value, 10) / 100;
  const threshAbs  = threshFrac * maxR;

  /* Paint the response overlay on top of the checkerboard */
  const out = ctx.createImageData(W, H);
  for (let i = 0; i < W * H; i++) {
    const r = R[i];
    if (r > threshAbs) {
      /* Red for confirmed corners above threshold */
      out.data[i*4]   = 230;
      out.data[i*4+1] = 57;
      out.data[i*4+2] = 70;
      out.data[i*4+3] = 200;
    } else if (r > 0) {
      /* Blue tint for edge-like positive responses below threshold */
      const t = r / maxR;
      out.data[i*4]   = Math.round(69 + t * 60);
      out.data[i*4+1] = Math.round(123 + t * 40);
      out.data[i*4+2] = Math.round(157 + t * 30);
      out.data[i*4+3] = Math.round(t * 160 + 20);
    } else {
      /* Transparent for flat or negative response */
      out.data[i*4+3] = 0;
    }
  }
  ctx.putImageData(out, 0, 0);

  /* Count detected corner pixels */
  let corners = 0;
  for (let i = 0; i < R.length; i++) if (R[i] > threshAbs) corners++;
  statusEl.textContent = i18nFmt('Detected %n% corner pixel(s) above threshold.', { n: corners });
}

thresh.addEventListener('input', () => {
  threshVal.textContent = thresh.value;
  run();
});
cellSzSel.addEventListener('change', run);
document.getElementById('btnRun').addEventListener('click', run);

run();

<!-- Harris Corner Detection demo — checkerboard with overlay -->
<div class="controls">
  <label>Threshold: <span id="threshVal">40</span>%
    <input type="range" id="thresh" min="1" max="99" value="40">
  </label>
  <label>Cell size:
    <select id="cellSz">
      <option value="16">16 px</option>
      <option value="24" selected>24 px</option>
      <option value="32">32 px</option>
    </select>
  </label>
  <button id="btnRun" type="button">Redraw</button>
</div>
<div class="canvas-wrap">
  <canvas id="cvs"></canvas>
</div>
<div class="legend">
  <span class="dot corner"></span> Corner
  <span class="dot edge"></span> Edge / gradient
  <span class="dot flat"></span> Flat
</div>
<div id="status" class="status"></div>

/* Styles for the Harris corner detection demo */
* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; margin: 0; color: #222; }
.controls { display: flex; gap: .7rem; flex-wrap: wrap; align-items: center; margin-bottom: .6rem; }
label { font-size: .88rem; display: flex; align-items: center; gap: .35rem; }
input[type=range] { width: 110px; cursor: pointer; }
select { font-size: .88rem; border: 1px solid #aaa; border-radius: 6px; padding: .15rem .3rem; }
button { font: 600 13px system-ui; padding: .4rem .85rem; border: 1px solid #1d3557;
         background: #1d3557; color: #fff; border-radius: 8px; cursor: pointer; }
.canvas-wrap { overflow: auto; max-width: 100%; }
canvas { display: block; image-rendering: pixelated; border: 1px solid #ccc; border-radius: 4px; }
.legend { font-size: .82rem; margin-top: .5rem; display: flex; gap: .9rem; align-items: center; }
.dot { display: inline-block; width: 12px; height: 12px; border-radius: 50%; }
.dot.corner { background: #e63946; }
.dot.edge    { background: #457b9d; }
.dot.flat    { background: #ccc; }
.status { font-size: .88rem; font-weight: 600; margin-top: .4rem; min-height: 1.2em; color: #1d3557; }

/* Harris Corner Detection demo — structure tensor on a checkerboard */
const i18nFmt = (s, v) => s.replace(/%(\w+)%/g, (_, k) => String(v[k] ?? ''));

const cvs    = document.getElementById('cvs');
const ctx    = cvs.getContext('2d');
const thresh = document.getElementById('thresh');
const threshVal = document.getElementById('threshVal');
const cellSzSel = document.getElementById('cellSz');
const statusEl  = document.getElementById('status');

/* Draw a black-and-white checkerboard on the canvas */
function drawCheckerboard(cellPx, cols, rows) {
  cvs.width  = cols * cellPx;
  cvs.height = rows * cellPx;
  for (let r = 0; r < rows; r++) {
    for (let c = 0; c < cols; c++) {
      ctx.fillStyle = (r + c) % 2 === 0 ? '#ffffff' : '#222222';
      ctx.fillRect(c * cellPx, r * cellPx, cellPx, cellPx);
    }
  }
}

/* Compute Sobel gradients Ix and Iy for every pixel */
function sobelGradients(img, W, H) {
  const d = img.data;
  const Ix = new Float32Array(W * H);
  const Iy = new Float32Array(W * H);
  const gray = new Float32Array(W * H);

  /* Convert to grayscale using luminance weights */
  for (let i = 0; i < W * H; i++) {
    gray[i] = 0.299 * d[i * 4] + 0.587 * d[i * 4 + 1] + 0.114 * d[i * 4 + 2];
  }

  /* Sobel 3x3 kernel for horizontal and vertical gradients */
  for (let y = 1; y < H - 1; y++) {
    for (let x = 1; x < W - 1; x++) {
      const idx = y * W + x;
      Ix[idx] =
        -gray[(y-1)*W+(x-1)] + gray[(y-1)*W+(x+1)]
        - 2*gray[y*W+(x-1)]  + 2*gray[y*W+(x+1)]
        - gray[(y+1)*W+(x-1)] + gray[(y+1)*W+(x+1)];
      Iy[idx] =
        -gray[(y-1)*W+(x-1)] - 2*gray[(y-1)*W+x] - gray[(y-1)*W+(x+1)]
        + gray[(y+1)*W+(x-1)] + 2*gray[(y+1)*W+x] + gray[(y+1)*W+(x+1)];
    }
  }
  return { Ix, Iy };
}

/* Compute Harris corner response R = det(M) - k * trace(M)^2 for each pixel */
function harrisResponse(Ix, Iy, W, H, winR, k) {
  const R = new Float32Array(W * H);
  for (let y = winR; y < H - winR; y++) {
    for (let x = winR; x < W - winR; x++) {
      let Sxx = 0, Sxy = 0, Syy = 0;
      /* Accumulate the 2x2 structure tensor over the window */
      for (let dy = -winR; dy <= winR; dy++) {
        for (let dx = -winR; dx <= winR; dx++) {
          const i = (y + dy) * W + (x + dx);
          Sxx += Ix[i] * Ix[i];
          Sxy += Ix[i] * Iy[i];
          Syy += Iy[i] * Iy[i];
        }
      }
      /* R = det - k * trace^2 (Harris 1988) */
      const det   = Sxx * Syy - Sxy * Sxy;
      const trace = Sxx + Syy;
      R[y * W + x] = det - k * trace * trace;
    }
  }
  return R;
}

/* Main run function — draw checkerboard then compute and overlay Harris response */
function run() {
  const cellPx = parseInt(cellSzSel.value, 10);
  const cols = 10, rows = 10;

  drawCheckerboard(cellPx, cols, rows);
  const W = cvs.width, H = cvs.height;
  const img = ctx.getImageData(0, 0, W, H);

  const { Ix, Iy } = sobelGradients(img, W, H);
  const R = harrisResponse(Ix, Iy, W, H, 2, 0.04);

  /* Find the max positive response for normalization */
  let maxR = 0;
  for (let i = 0; i < R.length; i++) if (R[i] > maxR) maxR = R[i];

  /* Absolute threshold from the slider percentage */
  const threshFrac = parseInt(thresh.value, 10) / 100;
  const threshAbs  = threshFrac * maxR;

  /* Paint the response overlay on top of the checkerboard */
  const out = ctx.createImageData(W, H);
  for (let i = 0; i < W * H; i++) {
    const r = R[i];
    if (r > threshAbs) {
      /* Red for confirmed corners above threshold */
      out.data[i*4]   = 230;
      out.data[i*4+1] = 57;
      out.data[i*4+2] = 70;
      out.data[i*4+3] = 200;
    } else if (r > 0) {
      /* Blue tint for edge-like positive responses below threshold */
      const t = r / maxR;
      out.data[i*4]   = Math.round(69 + t * 60);
      out.data[i*4+1] = Math.round(123 + t * 40);
      out.data[i*4+2] = Math.round(157 + t * 30);
      out.data[i*4+3] = Math.round(t * 160 + 20);
    } else {
      /* Transparent for flat or negative response */
      out.data[i*4+3] = 0;
    }
  }
  ctx.putImageData(out, 0, 0);

  /* Count detected corner pixels */
  let corners = 0;
  for (let i = 0; i < R.length; i++) if (R[i] > threshAbs) corners++;
  statusEl.textContent = i18nFmt('Detected %n% corner pixel(s) above threshold.', { n: corners });
}

thresh.addEventListener('input', () => {
  threshVal.textContent = thresh.value;
  run();
});
cellSzSel.addEventListener('change', run);
document.getElementById('btnRun').addEventListener('click', run);

run();

Result

Console

Drag the threshold slider to control how selective the detector is. Lower the threshold and more corners light up — including some that are barely distinguishable from edges. Raise it and only the sharpest corners survive. This trade-off between recall (finding everything) and precision (finding only true corners) is the same one that every feature detector faces.

The Structure Tensor

The core idea is the structure tensor (also called the second-moment matrix). For each pixel, compute the image gradients $I_x$ and $I_y$ — roughly, how fast brightness changes horizontally and vertically. Then form the $2 \times 2$ matrix

M = \begin{pmatrix} \sum I_x^2 & \sum I_x I_y \\ \sum I_x I_y & \sum I_y^2 \end{pmatrix}

where the sums run over a small window (typically $3 \times 3$ or $5 \times 5$ pixels) around the pixel of interest. The matrix $M$ encodes how the local image patch responds to shifts in each direction.

The eigenvalues $\lambda_1, \lambda_2$ of $M$ tell the whole story:

Both small ( $\lambda_1 \approx \lambda_2 \approx 0$ ): flat region — intensity barely changes anywhere.
One large, one small ( $\lambda_1 \gg \lambda_2$ ): edge — strong change in one direction, weak in another.
Both large ( $\lambda_1 \approx \lambda_2 \gg 0$ ): corner — strong change in every direction.

Computing eigenvalues explicitly is expensive. Harris and Stephens observed that you can score a pixel cheaply without them:

R = \det(M) - k \cdot \text{tr}(M)^2 = \lambda_1 \lambda_2 - k(\lambda_1 + \lambda_2)^2

with $k$ typically around $0.04$ – $0.06$ . A large positive $R$ means corner; $R$ near zero means flat; large negative $R$ means edge. The whole computation is $O(1)$ per pixel (one pass for gradients, one convolution for window sums, one arithmetic expression for $R$ ), so the full image runs in $O(n)$ time — linear in the number of pixels.

The detector was solved by Harris and Stephens in 1988 and remains in active use today, both directly and as the mathematical backbone of many modern pattern matching and dimensionality reduction pipelines.

Where It Matters

Stable, repeatable landmark points are the currency of computer vision. Harris corners appear wherever a system needs to track or match what it sees:

Panorama stitching: find matching corners in two overlapping photos, compute the homography between them, warp and blend. Apps like Google Photos and iPhone do this in milliseconds.
Structure from Motion (SfM): track corners across frames of a moving camera to reconstruct a 3-D scene from a flat video — the engine behind photogrammetry drones and city-scale 3-D maps.
SLAM (Simultaneous Localization and Mapping): robots and AR headsets use corners as landmarks to build a map while tracking their own position inside it.
Optical flow: matching corners between consecutive video frames gives the apparent motion of objects — used in video stabilization, sports analytics, and autonomous driving.
Augmented reality: stick a virtual object on a real-world corner and it stays anchored even as the camera moves.

Modern descriptors like SIFT, SURF, and ORB still detect corners at their core, just with more scale-space machinery around the Harris idea. The 1988 paper remains one of the most cited in all of computer vision.

Conclusion

Harris Corner Detection is a masterclass in turning a geometric intuition into a fast, practical algorithm. The observation that corners are "pinned" in every direction becomes a $2 \times 2$ matrix; the matrix becomes a scalar score; the score runs in linear time on the full image.

The corners it finds are not arbitrary pixels — they are the places where the image carries the most geometric information. Stitch them across two photographs and you can align the world. Track them across a video and you can reconstruct a 3-D scene from flat frames. That is the quiet power of a well-chosen mathematical structure: the right abstraction unlocks a whole family of hard problems at once.

Next time you open a panorama stitched perfectly by your phone, or watch an AR label stick to a real object through a moving camera, remember: it is anchored to a set of corners found by asking, for each tiny patch of pixels, how much does this place resist being confused with its neighbors?

Comments

Loading comments...

https://www.kipuhub.com/en/article/harris-corner-detection/Content licensed under CC BY-NC 4.0.

Cookies & privacy

How a 2×2 matrix tells flat from edge from corner