Convolutional Neural Networks

When you glance at a photo, your brain doesn't analyze every pixel in isolation — it senses edges, then textures, then shapes, then objects, all in a cascade of processing. For decades, engineers tried to program those steps by hand. They failed.

The breakthrough came from letting the machine learn those steps from data. Convolutional neural networks (CNNs) do exactly that: they pass a small grid of numbers — a filter — over the input image, measuring how much each patch looks like a particular pattern. Repeat with hundreds of different filters, stack several such layers, and the network builds a hierarchy: first edges, then corners, then eyes or wheels, then faces or cars.

The key insight is weight sharing: every position in the image is processed by the same filter. A filter that detects a horizontal edge works whether that edge is at the top or bottom of the image. This reduces the number of parameters enormously compared to a fully connected network, making it practical to learn from millions of images.

Yann LeCun and colleagues demonstrated the idea in 1989 on handwritten digit recognition. The explosion came in 2012 when AlexNet — a deep CNN trained on a GPU — slashed the ImageNet error rate by almost half in a single year, touching off the modern deep-learning era.

The canvas below shows a simple 8×8 pixel image (a bright cross on a dark background). A 3×3 edge-detecting filter (a discrete Laplacian) is applied at every position and the result is shown on the right.

<div class="demo-wrap">
  <div class="panel">
    <div class="label">{{lbl_input}} <span class="badge">8×8</span></div>
    <canvas id="imgCanvas" width="192" height="192"></canvas>
  </div>
  <div class="panel">
    <div class="label">{{lbl_filter}} <span class="badge">{{badge_edge}}</span></div>
    <canvas id="filterCanvas" width="72" height="72"></canvas>
    <div class="filter-vals" id="filterVals"></div>
  </div>
  <div class="panel">
    <div class="label">{{lbl_output}} <span class="badge">6×6</span></div>
    <canvas id="outCanvas" width="144" height="144"></canvas>
  </div>
</div>
<div class="controls">
  <label>{{lbl_filter_pos}}
    <input type="range" id="posSlider" min="0" max="35" value="0">
    <span id="posLabel">{{pos_row}} 0, {{pos_col}} 0</span>
  </label>
  <button id="scanBtn" type="button">{{btn_autoscan}}</button>
  <button id="resetBtn" type="button" class="ghost">{{btn_reset}}</button>
</div>
<div class="info" id="info">{{info_initial}}</div>

* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; margin: 0; color: #222; font-size: 14px; }
.demo-wrap { display: flex; gap: 16px; flex-wrap: wrap; align-items: flex-start; margin-bottom: 10px; }
.panel { display: flex; flex-direction: column; align-items: center; gap: 4px; }
.label { font-size: .78rem; font-weight: 600; color: #555; }
.badge { background: #e8eef3; color: #1d3557; border-radius: 4px; padding: 1px 5px; font-size: .72rem; font-weight: 500; margin-left: 4px; }
canvas { border: 1px solid #cdd9e3; border-radius: 6px; image-rendering: pixelated; display: block; }
.filter-vals { display: grid; grid-template-columns: repeat(3, 24px); gap: 2px; font: 600 11px ui-monospace, monospace; color: #1d3557; }
.filter-vals span { text-align: center; }
.controls { display: flex; flex-wrap: wrap; align-items: center; gap: 10px; margin-bottom: 6px; }
label { font-size: .85rem; color: #333; display: flex; align-items: center; gap: 6px; }
input[type=range] { width: 140px; }
#posLabel { font-size: .78rem; color: #666; min-width: 80px; }
button { font: 600 13px system-ui, sans-serif; padding: .35rem .75rem; border: 1px solid #1d3557;
         background: #1d3557; color: #fff; border-radius: 7px; cursor: pointer; }
button.ghost { background: #fff; color: #1d3557; }
.info { font-size: .85rem; color: #444; min-height: 1.4em; }

// Code not found

Drag the filter selector to move the active patch, or click Auto-scan to watch the filter slide across every position automatically. The output pixel brightens when the filter finds a strong match and darkens when it finds nothing. Notice that edges of the cross light up — exactly the feature a first CNN layer would learn to detect.

Stacking convolution layers is powerful, but it raises real questions about scale and trainability.

• Parameters: a convolution layer with 64 filters of size 3×3 has only 64 × 9 = 576 weights, regardless of image size. A fully connected layer over a 256×256 image would need billions. Weight sharing is what makes CNNs feasible.
• Depth and vanishing gradients: adding more layers lets the network learn more abstract features, but gradients shrink as they travel backward through many layers. The ResNet architecture (He et al., 2015) solved this with skip connections — adding the input of a block directly to its output — allowing networks hundreds of layers deep to train reliably.
• Pooling: after each convolution, a max-pooling layer shrinks the spatial size by keeping only the largest activation in each small region. This builds translation invariance and reduces computation.
• Batch normalization (Ioffe & Szegedy, 2015): normalizing activations at each layer dramatically accelerates training and reduces sensitivity to initialization.
• Training cost: a modern CNN like ResNet-50 has about 25 million parameters and needs hundreds of GPU-hours to train on ImageNet. Inference, however, is fast enough to run in real time on a phone.

CNNs are a solved engineering problem in the sense that we know how to train them. The open research questions are about why they generalize so well, what they actually learn, and whether they can be made robust to adversarial inputs — images deliberately perturbed to fool the network.

See also neural network training and machine learning generalization.

CNNs are the backbone of almost every modern visual system:

• Image classification: given a photo, name the main object. AlexNet, VGG, GoogLeNet, ResNet, EfficientNet — each generation pushed accuracy on ImageNet and became the transfer-learning base for everything else.
• Object detection: locate and label multiple objects in a scene. YOLO, Faster R-CNN, and SSD are CNN-based and run fast enough for video.
• Medical imaging: CNNs detect diabetic retinopathy from fundus photographs, tumors in CT scans, and anomalies in pathology slides — often matching specialist accuracy.
• Autonomous vehicles: road segmentation, lane detection, pedestrian recognition, and traffic-sign classification all rely on CNNs processing camera frames in real time.
• Face recognition: the unlock system on your phone applies a CNN to a face crop and compares the embedding to stored templates.
• Beyond vision: 1-D convolutions process audio waveforms (speech recognition) and DNA sequences (genomics). The same weight-sharing logic applies whenever the data has local structure that repeats across positions.

CNNs also underpin generative models: variational autoencoders and GANs use convolutional encoders and decoders to compress and reconstruct images, enabling style transfer, super-resolution, and image synthesis.

The elegance of convolutional networks is that a simple idea — slide a small, learned filter across an image — turns out to be sufficient to build a hierarchy of visual features that rivals human perception on benchmark tasks.

Weight sharing keeps the parameter count manageable. Depth lets the network compose simple features into complex ones. Skip connections and batch normalization make very deep networks trainable. And massive labeled datasets give the network the signal it needs to learn filters that generalize.

CNNs did not emerge from a theory that said "this is the right architecture for vision." They emerged from decades of experimentation, a biological analogy (the visual cortex), and the sudden availability of GPUs and data. That is a recurring theme in machine learning: the gap between "it works" and "we know why it works" remains large — which means neural network training and interpretability are still active research frontiers.