Introduction

Imagine a counterfeiter trying to pass off fake banknotes, and a detective whose only job is to tell real from fake. Each time the detective gets sharper, the forger is forced to improve. Each time the forger improves, the detective must look harder. Push that loop far enough and the fakes become indistinguishable from the real thing.

That is exactly the idea behind a Generative Adversarial Network (GAN), introduced by Ian Goodfellow and his co-authors in 2014. Two neural networks train against each other: a generator that turns random noise into samples, and a discriminator that scores how likely a sample is to be real. They are not collaborators — they are opponents in a minimax game.

The surprise is how much comes out of so little. Nobody hand-labels what "realistic" means. The discriminator's pushback is the learning signal, and from that single tug-of-war emerge faces, voices and images that never existed.

Watch the Arms Race

Below, the blue curve is a hidden target distribution — the "real data" the generator has never seen directly. The orange bars are the samples the generator is currently producing. A simple discriminator rates each region as more real or more fake, and that signal nudges the generator's samples toward the truth.

<p class="hint">The <b>blue curve</b> is a hidden target distribution the generator
never sees. The <b>orange bars</b> are the generator's current samples. The
discriminator scores each region real-vs-fake (gray line); that signal nudges
the samples toward the truth. Train until orange matches blue.</p>
<canvas id="plot" width="460" height="240"></canvas>
<div class="status" id="status">Round 0 — samples are random noise.</div>
<div class="btns">
  <button id="step" type="button">Train one round</button>
  <button id="auto" type="button">Auto-train</button>
  <button id="reset" type="button" class="ghost">Reset</button>
</div>

* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; color: #222; margin: 0; }
.hint { font-size: .9rem; color: #444; margin: 0 0 .7rem; line-height: 1.45; }
canvas { width: 100%; max-width: 460px; height: auto; background: #f7f9fb;
         border: 1px solid #cdd9e3; border-radius: 8px; }
.status { font-size: 1rem; font-weight: 600; margin: .6rem 0; min-height: 1.4em; }
.status.ok { color: #0a7d33; }
.btns { display: flex; gap: .5rem; flex-wrap: wrap; }
button { font: 600 14px system-ui, sans-serif; padding: .45rem .9rem; border: 1px solid #1d3557;
         background: #1d3557; color: #fff; border-radius: 8px; cursor: pointer; }
button.ghost { background: #fff; color: #1d3557; }

const fmt = (s, v) => s.replace(/%(\w+)%/g, (_, k) => String(v[k] ?? ''));
// A tiny 1-D GAN intuition. The "real data" is a fixed hidden target (a
    // Gaussian bump). The generator is a cloud of sample points on [0,1].
    // Each round the DISCRIMINATOR estimates, per region, how real-vs-fake it is;
    // its log-odds equal log p_real - log p_fake. The GENERATOR then moves each
    // sample UPHILL on that score, with calibrated noise, so its samples settle
    // into exactly the real distribution (not collapsed onto the single peak).
    // Verified: across seeds the orange histogram converges to the blue curve
    // (sample mean -> target mean, sample spread -> target spread).
const cv = document.getElementById('plot'), ctx = cv.getContext('2d');
const W = cv.width, H = cv.height, PAD = 24;
const MU = 0.62, SIG = 0.10;                 // hidden target: Gaussian bump
const real = x => Math.exp(-((x-MU)**2)/(2*SIG*SIG));
const NB = 24;                               // bins for histograms / scoring
const N = 240;                               // generator sample count
const EPS = 0.002;                           // generator step size
let G = [], logp = [], round = 0, timer = null;

function realHist(){ const h=new Array(NB).fill(0);
  for(let k=0;k<NB;k++) h[k]=real((k+0.5)/NB);
  const t=h.reduce((a,c)=>a+c,0)||1; return h.map(c=>c/t); }
function hist(s){ const h=new Array(NB).fill(0);
  s.forEach(x=>{ let k=Math.min(NB-1,(x*NB)|0); h[k]++; });
  const t=h.reduce((a,c)=>a+c,0)||1; return h.map(c=>c/t); }
function err(){ const a=hist(G), r=realHist();
  return a.reduce((acc,v,i)=>acc+Math.abs(v-r[i]),0)/2; } // total variation

function init(){
  G = Array.from({length:N}, () => Math.random());   // pure noise
  logp = new Array(NB).fill(0); round = 0;
  stop(); setStatus('Round 0 — samples are random noise.',''); draw();
}
function gauss(){ const u=Math.random(), v=Math.random();
  return Math.sqrt(-2*Math.log(u))*Math.cos(2*Math.PI*v); }

function trainStep(){
  // Discriminator: smooth running estimate of the real log-density per bin.
  const r = realHist();
  for(let k=0;k<NB;k++) logp[k] = 0.7*logp[k] + 0.3*Math.log(r[k]+1e-4);
  // Generator: Langevin step uphill on the score, x += eps*grad + sqrt(2 eps)*noise.
  const dx = 1/NB;
  const Lat = x => { let k=Math.min(NB-1,Math.max(0,(x*NB)|0)); return logp[k]; };
  for(let i=0;i<N;i++){ const x=G[i];
    const grad = (Lat(x+dx)-Lat(x-dx))/(2*dx);
    G[i] += EPS*grad + Math.sqrt(2*EPS)*gauss();
    if(G[i]<0) G[i]=-G[i]; if(G[i]>1) G[i]=2-G[i];   // reflect at walls
    if(G[i]<0) G[i]=0; if(G[i]>1) G[i]=1;
  }
  round++;
  const e = err();
  if(e < 0.07){ setStatus(fmt('Round %round% — converged: fakes match the target!', { round: round }),'ok'); stop(); }
  else setStatus(fmt('Round %round% — mismatch %e%, still chasing.', { round: round, e: e.toFixed(2) }),'');
  draw();
}

function draw(){
  ctx.clearRect(0,0,W,H);
  const x2 = x => PAD + x*(W-2*PAD), y2 = v => H-PAD - v*(H-2*PAD);
  const r = realHist(), rmax = Math.max(...r);
  const a = hist(G), amax = Math.max(...a, rmax);
  // generator histogram (orange bars)
  ctx.fillStyle = '#f4a23688';
  for(let k=0;k<NB;k++){ const bx=x2(k/NB), bw=(W-2*PAD)/NB-1;
    ctx.fillRect(bx, y2(a[k]/amax), bw, (H-PAD)-y2(a[k]/amax)); }
  // real target curve (blue)
  const peak = Math.max(...Array.from({length:121},(_,j)=>real(j/120)));
  ctx.strokeStyle='#1d6fd1'; ctx.lineWidth=2.5; ctx.beginPath();
  for(let i=0;i<=120;i++){ const x=i/120, v=real(x)/peak;
    const px=x2(x), py=y2(v); i?ctx.lineTo(px,py):ctx.moveTo(px,py); }
  ctx.stroke();
  // discriminator score (gray dashed) — normalised log-density
  const lo=Math.min(...logp), hi=Math.max(...logp), span=(hi-lo)||1;
  ctx.strokeStyle='#9aa3ad'; ctx.lineWidth=1.5; ctx.setLineDash([4,3]); ctx.beginPath();
  for(let k=0;k<NB;k++){ const px=x2((k+0.5)/NB), py=y2((logp[k]-lo)/span);
    k?ctx.lineTo(px,py):ctx.moveTo(px,py); }
  ctx.stroke(); ctx.setLineDash([]);
  ctx.fillStyle='#666'; ctx.font='11px system-ui';
  ctx.fillText('real', x2(MU)-12, y2(1)+12);
  ctx.fillText('discriminator', x2(0.04), y2(0.04));
}
function setStatus(t,cls){ const s=document.getElementById('status'); s.textContent=t; s.className='status '+cls; }
function stop(){ if(timer){ clearInterval(timer); timer=null; document.getElementById('auto').textContent='Auto-train'; } }
document.getElementById('step').addEventListener('click',trainStep);
document.getElementById('auto').addEventListener('click',()=>{
  if(timer) stop();
  else { document.getElementById('auto').textContent='Stop'; timer=setInterval(trainStep,160); }
});
document.getElementById('reset').addEventListener('click',init);
init();
Result
Console

Press Train one round to run a single adversarial step, or Auto-train to let the loop run. Watch the orange samples slide and reshape until they sit on top of the blue curve — the moment the discriminator can no longer tell them apart. That convergence, where neither network can improve, is the equilibrium a GAN is chasing.

The Real Difficulty

A GAN's elegance hides a genuinely hard problem. This is not a question that was proven impossible or settled — training GANs reliably remains an open, active research challenge.

  • It is a minimax game, not a single objective. The generator minimizes what the discriminator maximizes. You are not climbing toward one summit; you are looking for a saddle point — a Nash equilibrium where neither side can do better unilaterally.
  • Equilibria are hard to reach. Two networks improving at once can oscillate forever, chase each other in circles, or have one overpower the other so badly the learning signal vanishes.
  • Mode collapse. The generator can cheat by producing one convincing sample over and over, fooling the discriminator while ignoring most of the real variety.
  • No clean stopping rule. Unlike ordinary neural network training, a falling "loss" does not mean a better GAN — the target is moving because the opponent is learning too.

Goodfellow's 2014 paper proved that if the networks were arbitrarily powerful and trained perfectly, the generator would exactly recover the true data distribution. Real networks are neither, which is why a decade of work — Wasserstein GANs, gradient penalties, spectral normalization — has gone into merely making the duel stable.

Where It Matters

"Learn to produce data that passes for real" turns out to be enormously useful, and GANs were the breakthrough that made it practical:

  • Photorealistic images: GANs produced the first uncanny "this person does not exist" faces and high-fidelity synthetic scenes.
  • Super-resolution and restoration: turning blurry, low-resolution images into sharp ones, and reviving old photos.
  • Data augmentation: generating extra training examples for fields — like medical imaging — where real labeled data is scarce.
  • Art, style and design: translating sketches to photos, day to night, or one artist's style onto another image.
  • Simulation and privacy: synthesizing realistic-but-fake records so systems can be tested without exposing real personal data.

The same adversarial idea — a critic that gets the generator to improve — echoes through much of modern generative AI, sitting alongside neural network training as one of the field's foundational tricks.

Conclusion

GANs took a deceptively simple idea — let two networks fight — and turned it into one of the most influential ideas in modern AI. The generator never sees the real data directly; it learns entirely from the discriminator's verdicts, sculpting noise into something that looks true.

But the same duel that makes GANs powerful makes them fragile. Balancing two adversaries is a delicate, still-unsolved optimization game, full of collapses and oscillations. The forger and the detective will keep training each other — and keeping that fight fair remains one of the liveliest open problems in machine learning.

Share this article

Pick a channel — or use your device's native share sheet.

Comments

Loading comments...

https://www.kipuhub.com/en/article/gans/Content licensed under CC BY-NC 4.0.