Introduction

In 2014, a group of researchers at Google Brain made a disturbing discovery. A neural network that could identify cats, dogs, and school buses with superhuman accuracy could be tricked โ€” not by drawing a cartoon, not by adding noise, but by changing individual pixel values by amounts too small for a human eye to detect. The altered image looked identical to the original. The network, however, was suddenly certain it was looking at something else entirely.

These crafted inputs are called adversarial examples. They expose a profound gap between how neural networks learn and how humans see. A network builds a statistical shortcut through a high-dimensional space of pixel values; humans build a semantic model of objects. These two representations live in the same pixel space but carve it up in radically different ways.

The phenomenon is not a curiosity confined to toy networks. It affects the best classifiers we have, across images, audio, and text. And the perturbations can often be made to transfer: an attack crafted against one network fools an entirely different network trained separately. That transferability is what turns a neat theoretical result into a genuine security threat.

Nudge the Pixels

The demo below simulates the Fast Gradient Sign Method (FGSM), introduced by Goodfellow et al. in 2015. FGSM computes the gradient of the loss with respect to the input image โ€” in other words, the direction in pixel space that would make the current prediction worse โ€” and then nudges every pixel by a tiny amount ฮต in exactly that direction.

<div class="demo-wrap">
  <div class="panel">
    <div class="panel-label">Original image</div>
    <canvas id="origCanvas" width="140" height="140"></canvas>
    <div class="pred" id="origPred"></div>
  </div>
  <div class="panel">
    <div class="panel-label">Perturbation (ร—20)</div>
    <canvas id="pertCanvas" width="140" height="140"></canvas>
    <div class="pred small" id="pertInfo">scaled noise</div>
  </div>
  <div class="panel">
    <div class="panel-label">Adversarial image</div>
    <canvas id="advCanvas" width="140" height="140"></canvas>
    <div class="pred" id="advPred"></div>
  </div>
</div>
<div class="controls">
  <label for="eps">Perturbation ฮต = <span id="epsVal">0.00</span></label>
  <input type="range" id="eps" min="0" max="0.30" step="0.01" value="0">
</div>
<div class="bar-section">
  <div class="bar-label">Classifier confidence</div>
  <div class="bar-row"><span class="bar-name" id="cls0name">Cat</span><div class="bar-bg"><div class="bar-fill cat" id="bar0"></div></div><span class="bar-pct" id="pct0">โ€”</span></div>
  <div class="bar-row"><span class="bar-name" id="cls1name">Dog</span><div class="bar-bg"><div class="bar-fill dog" id="bar1"></div></div><span class="bar-pct" id="pct1">โ€”</span></div>
</div>
<div class="note" id="noteBox"></div>

* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; color: #222; margin: 0; padding: .5rem; }
.demo-wrap { display: flex; gap: 12px; flex-wrap: wrap; }
.panel { display: flex; flex-direction: column; align-items: center; }
.panel-label { font-size: .75rem; font-weight: 600; color: #555; margin-bottom: 4px; text-transform: uppercase; letter-spacing: .04em; }
canvas { border: 1.5px solid #cdd9e3; border-radius: 6px; image-rendering: pixelated; }
.pred { font-size: .85rem; font-weight: 700; margin-top: 5px; min-height: 1.2em; text-align: center; }
.pred.small { font-size: .72rem; color: #666; font-weight: 400; }
.controls { margin: .9rem 0 .4rem; }
.controls label { font-size: .85rem; font-weight: 600; display: block; margin-bottom: 4px; }
input[type=range] { width: 100%; max-width: 360px; }
.bar-section { margin-top: .6rem; }
.bar-label { font-size: .75rem; font-weight: 600; color: #555; margin-bottom: 6px; text-transform: uppercase; letter-spacing: .04em; }
.bar-row { display: flex; align-items: center; gap: 8px; margin-bottom: 6px; }
.bar-name { font-size: .8rem; font-weight: 600; width: 30px; }
.bar-bg { flex: 1; background: #e8eef3; border-radius: 4px; height: 18px; overflow: hidden; max-width: 260px; }
.bar-fill { height: 100%; border-radius: 4px; transition: width .1s; width: 0%; }
.bar-fill.cat { background: #2a7ae2; }
.bar-fill.dog { background: #e63946; }
.bar-pct { font-size: .8rem; font-weight: 700; width: 40px; }
.note { font-size: .82rem; margin-top: .7rem; padding: .55rem .75rem; border-radius: 6px; background: #f0f4f8; color: #333; min-height: 2.5em; line-height: 1.45; }
.note.flip { background: #fdecea; color: #b02020; font-weight: 600; }
.note.warn { background: #fff8e1; color: #7a5900; }

const fmt = (s, v) => s.replace(/%(\w+)%/g, (_, k) => String(v[k] ?? ''));
// Adversarial Examples demo โ€” simulates FGSM on a tiny linear classifier
// operating on a 7x7 "image" (49 pixels, each โˆˆ [0,1]).
//
// The classifier distinguishes a "cat" pattern (bright center ring) from a
// "dog" pattern (bright corners). Its weights are hand-crafted to match the
// geometry: w_cat[i] > 0 where a cat pixel is bright, w_dog[i] > 0 where a
// dog pixel is bright. The score for class k is w_k ยท x + b_k.
// FGSM: adversarial_x = x + ฮต ยท sign(โˆ‡_x L)
// For a 2-class softmax, โˆ‡_x L (targeting wrong class) = sign(w_dog - w_cat).
// This is verifiable with node: see the console.log assertions below.

const N = 7; // 7x7 grid = 49 pixels
const PIXELS = N * N;
const SCALE = 140 / N; // canvas scale (20px per grid cell)

// โ”€โ”€ Original image: "cat" โ€” bright ring at radius 2, dark corners โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
function makeImage() {
  const img = new Float32Array(PIXELS);
  const cx = (N - 1) / 2, cy = (N - 1) / 2;
  for (let r = 0; r < N; r++) for (let c = 0; c < N; c++) {
    const d = Math.sqrt((r - cy) ** 2 + (c - cx) ** 2);
    // bright ring at d โ‰ˆ 2, dark center and corners
    img[r * N + c] = Math.max(0, 1 - Math.abs(d - 2) * 0.9);
  }
  return img;
}

// โ”€โ”€ Classifier weights โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
// w_cat: positive where the ring is (d โ‰ˆ 2), negative elsewhere
// w_dog: positive at the four corners (d > 3.5), negative elsewhere
function makeWeights() {
  const wCat = new Float32Array(PIXELS);
  const wDog = new Float32Array(PIXELS);
  const cx = (N - 1) / 2, cy = (N - 1) / 2;
  for (let r = 0; r < N; r++) for (let c = 0; c < N; c++) {
    const d = Math.sqrt((r - cy) ** 2 + (c - cx) ** 2);
    const i = r * N + c;
    wCat[i] = Math.max(0, 1 - Math.abs(d - 2) * 0.9) * 0.4 - 0.10;
    wDog[i] = (d > 3.2 ? 2.0 : -0.2);
  }
  return { wCat, wDog };
}

// dot product
function dot(a, b) { let s = 0; for (let i = 0; i < a.length; i++) s += a[i] * b[i]; return s; }

// softmax over 2 logits
function softmax2(s0, s1) {
  const m = Math.max(s0, s1);
  const e0 = Math.exp(s0 - m), e1 = Math.exp(s1 - m);
  const z = e0 + e1;
  return [e0 / z, e1 / z];
}

// clamp to [0,1]
function clamp01(v) { return Math.max(0, Math.min(1, v)); }

// โ”€โ”€ FGSM gradient: sign(w_dog - w_cat) (pushes toward "dog") โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
function fgsmSign(wCat, wDog) {
  const g = new Float32Array(PIXELS);
  for (let i = 0; i < PIXELS; i++) g[i] = Math.sign(wDog[i] - wCat[i]);
  return g;
}

// โ”€โ”€ Draw a Float32Array image onto a canvas โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
function drawImage(canvas, img, tint) {
  const ctx = canvas.getContext('2d');
  ctx.clearRect(0, 0, canvas.width, canvas.height);
  for (let r = 0; r < N; r++) for (let c = 0; c < N; c++) {
    const v = img[r * N + c];
    const bright = Math.round(clamp01(v) * 255);
    if (tint === 'noise') {
      // show perturbation: positive = blue, negative = red
      const pos = Math.round(Math.max(0, v) * 255);
      const neg = Math.round(Math.max(0, -v) * 255);
      ctx.fillStyle = `rgb(${neg},${Math.round(bright * 0.5)},${pos})`;
    } else {
      ctx.fillStyle = `rgb(${bright},${bright},${bright})`;
    }
    ctx.fillRect(c * SCALE, r * SCALE, SCALE, SCALE);
  }
}

// โ”€โ”€ Initialise โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
const orig = makeImage();
const { wCat, wDog } = makeWeights();
const grad = fgsmSign(wCat, wDog);

// Verify: original image should classify as "cat" (score_cat > score_dog)
// Expected: cat โ‰ˆ 2.677, dog โ‰ˆ -2.802 (verified with node)
const s0orig = dot(wCat, orig), s1orig = dot(wDog, orig);
console.assert(s0orig > s1orig,
  fmt('VERIFY: cat score (%s0orig%) should exceed dog score (%s1orig%)', { s0orig: s0orig.toFixed(3), s1orig: s1orig.toFixed(3) }));

// Verify: with ฮต=0.20 the adversarial example should flip to "dog"
// Expected: cat โ‰ˆ 1.828, dog โ‰ˆ 2.958 (verified with node)
const EPS_TEST = 0.20;
const advTest = new Float32Array(PIXELS);
for (let i = 0; i < PIXELS; i++) advTest[i] = clamp01(orig[i] + EPS_TEST * grad[i]);
const s0adv = dot(wCat, advTest), s1adv = dot(wDog, advTest);
console.assert(s1adv > s0adv,
  fmt(`VERIFY: at eps=0.20 dog score (%s1adv%) should exceed cat score (%s0adv%)`, { s1adv: s1adv.toFixed(3), s0adv: s0adv.toFixed(3) }));
console.log(`Scores original โ€” cat:`, s0orig.toFixed(3), `dog:`, s1orig.toFixed(3));
console.log(`Scores adv(ฮต=0.20) โ€” cat:`, s0adv.toFixed(3), `dog:`, s1adv.toFixed(3));

drawImage(document.getElementById('origCanvas'), orig, 'gray');

const epsSlider = document.getElementById('eps');
const epsVal = document.getElementById('epsVal');

function update() {
  const eps = parseFloat(epsSlider.value);
  epsVal.textContent = eps.toFixed(2);

  // build adversarial image
  const adv = new Float32Array(PIXELS);
  const noise = new Float32Array(PIXELS);
  for (let i = 0; i < PIXELS; i++) {
    noise[i] = eps * grad[i]; // true perturbation
    adv[i] = clamp01(orig[i] + noise[i]);
  }

  // draw noise (amplified ร—20 for visibility)
  const noiseViz = new Float32Array(PIXELS);
  for (let i = 0; i < PIXELS; i++) noiseViz[i] = noise[i] * 20;
  drawImage(document.getElementById('pertCanvas'), noiseViz, 'noise');
  drawImage(document.getElementById('advCanvas'), adv, 'gray');

  // classify original
  const [p0o, p1o] = softmax2(dot(wCat, orig), dot(wDog, orig));
  const origLabel = p0o > p1o ? `Cat` : `Dog`;
  const origEl = document.getElementById('origPred');
  origEl.textContent = origLabel + ' (' + Math.round(Math.max(p0o, p1o) * 100) + '%)';
  origEl.style.color = origLabel === `Cat` ? '#2a7ae2' : '#e63946';

  // classify adversarial
  const [p0a, p1a] = softmax2(dot(wCat, adv), dot(wDog, adv));
  const advLabel = p0a > p1a ? `Cat` : `Dog`;
  const advEl = document.getElementById('advPred');
  advEl.textContent = advLabel + ' (' + Math.round(Math.max(p0a, p1a) * 100) + '%)';
  advEl.style.color = advLabel === `Cat` ? '#2a7ae2' : '#e63946';

  // confidence bars (show adversarial confidence)
  const bar0 = document.getElementById('bar0');
  const bar1 = document.getElementById('bar1');
  bar0.style.width = Math.round(p0a * 100) + '%';
  bar1.style.width = Math.round(p1a * 100) + '%';
  document.getElementById('pct0').textContent = Math.round(p0a * 100) + '%';
  document.getElementById('pct1').textContent = Math.round(p1a * 100) + '%';

  // note
  const note = document.getElementById('noteBox');
  if (eps === 0) {
    note.textContent = `No perturbation. The classifier correctly identifies the pattern as Cat with high confidence.`;
    note.className = 'note';
  } else if (advLabel !== origLabel) {
    note.textContent = fmt(`Prediction flipped to %advLabel%! The image looks identical to you, yet the classifier is now wrong with %round%% confidence.`, { advLabel: advLabel, round: Math.round(Math.max(p0a, p1a) * 100) });
    note.className = 'note flip';
  } else {
    note.textContent = fmt(`Perturbation applied but prediction still holds at ฮต = %eps%. Try increasing ฮต.`, { eps: eps.toFixed(2) });
    note.className = 'note warn';
  }
}

epsSlider.addEventListener('input', update);
update();
Result
Console

Notice that the perturbed image looks the same as the original to your eye. Yet the classifier's confidence in the true label plummets and confidence in a wrong label spikes. Increase ฮต and the attack becomes almost certain โ€” but the noise finally starts to look grainy. The sweet spot where the attack is both invisible and lethal is exactly the adversarial regime.

The Real Complexity

Why do adversarial examples exist at all? The answer lives in high-dimensional geometry.

  • The curse of dimensionality. A 224ร—224 RGB image lives in a space with roughly 150 000 dimensions. In such spaces almost all volume sits near the boundary of any region, not at its center. A classifier that carves out a "cat region" faces an enormous boundary โ€” and adversarial examples live just across that boundary.
  • Linear models are already vulnerable. Goodfellow et al. showed that even a purely linear classifier with weights w and input x has output wยทx + b. Perturb x by ฮตยทsign(w): the change to the output is ฮตยทโ€–wโ€–โ‚, which can be enormous when the dimension is large. Neural networks, being locally near-linear, inherit this.
  • Certified robustness is hard. Proving that no perturbation of size ฮต can flip a prediction is co-NP-hard in general. Practical defenses use randomized smoothing, adversarial training, or certified bounds โ€” each with real costs in accuracy or compute. No free lunch.
  • Status: active open problem. There is no proven method that makes large neural networks both accurate and certifiably robust at scale. The 2017โ€“2019 Madry adversarial training framework remains a gold standard, but it trades clean accuracy for robustness. Closing that gap is one of the central open questions in neural-network training research.

The key takeaway: adversarial fragility is not a programming bug to patch. It is a consequence of the geometry of high-dimensional function approximation, and taming it requires fundamentally rethinking how we measure a model's success.

Where It Matters

Adversarial examples matter wherever a neural network makes a decision that costs something if it is wrong:

  • Autonomous vehicles: stop-sign stickers, lane-marking tape, and spoofed LiDAR returns have all been shown to cause misclassification in real perception pipelines. A patch on a road can make a car ignore a pedestrian.
  • Medical imaging: adversarial perturbations on X-rays and MRI scans can flip a "benign" diagnosis to "malignant" or vice versa, with no visible change in the image. Clinical deployment of AI diagnostics must account for this.
  • Face recognition and authentication: slight glasses frames or face-paint patterns have been shown to fool face-ID systems, enabling impersonation attacks.
  • Content moderation: adversarial text and image perturbations can bypass hate-speech, CSAM, and spam classifiers at scale, evading filters that cost platforms enormous resources to build.
  • Malware detection: adversarial perturbations to binary features can make malicious code look benign to ML-based antivirus systems.

The field of adversarial machine learning now covers attacks (white-box, black-box, physical-world), defenses (adversarial training, certified bounds, input preprocessing), and evaluations. Robustness benchmarks like RobustBench track the state of the art.

Closely related is the broader question of what neural-network training actually optimizes and what guarantees that optimization gives โ€” a question without a clean answer today.

Conclusion

A network that recognizes cats across millions of photos can be confused by a pattern invisible to any human. That is not a bug in one product โ€” it is a structural property of how high-dimensional statistical models partition space.

Adversarial examples have forced the field to ask harder questions: What does it mean for a model to understand an image rather than merely match its statistics? What guarantees do we need before deploying a classifier in a safety-critical system? And can we ever train a model that is both maximally accurate and provably robust?

None of those questions have satisfying answers yet. In the meantime, every self-driving car, every medical AI, and every content-moderation system lives with adversarial fragility as a known, unresolved risk. The invisible pixel has turned out to be one of the most important findings in modern machine learning.

Share this article

Pick a channel โ€” or use your device's native share sheet.

Comments

Loading comments...

https://www.kipuhub.com/en/article/adversarial-examples/Content licensed under CC BY-NC 4.0.