Introduction

Most machine learning training starts with a fixed blueprint: pick a number of layers, choose how many neurons each layer has, and then adjust the weights until the network performs well. The architecture is decided by the engineer; only the numbers inside it are learned.

NEAT — NeuroEvolution of Augmenting Topologies — takes a different path. Introduced by Kenneth O. Stanley and Risto Miikkulainen in 2002, it treats the structure of a neural network as something that can itself be evolved. A NEAT population starts with the simplest possible networks — inputs wired directly to outputs, nothing in between — and mutations gradually add neurons and connections over generations.

Three ideas make this work without collapsing into chaos:

  • Innovation numbers tag every new gene (node or connection) with a global timestamp, so two networks that independently invented the same structure can be recognised and crossed over correctly.
  • Speciation groups similar individuals together so that a new, structurally different individual is not immediately out-competed before it has a chance to refine its novel structure.
  • Complexification from minimal starts means the search begins in the smallest possible space and grows only when complexity earns its keep.

The result is an algorithm that explores the space of both weights and architectures simultaneously, discovering efficient network topologies the engineer never specified.

Watch a Network Grow

The demo below runs a simplified NEAT loop on a tiny XOR-like task. Each node in the network is shown as a circle; connections are lines whose thickness reflects their weight strength. Press Evolve one generation to apply mutations and selection, and watch the topology grow.

<!-- NEAT neuroevolution demo — network topology viewer -->
<div class="panel">
  <div class="controls">
    <button id="btn-evolve" type="button">Evolve one generation</button>
    <button id="btn-reset" type="button" class="ghost">Reset</button>
    <span class="gen-label">Generation: <strong id="gen-num">0</strong></span>
  </div>
  <div class="status-row">
    <span class="badge" id="badge-fitness">Best fitness: <strong id="fit-val">0.00</strong></span>
    <span class="badge" id="badge-nodes">Nodes: <strong id="node-val">0</strong></span>
    <span class="badge" id="badge-conns">Connections: <strong id="conn-val">0</strong></span>
  </div>
</div>
<canvas id="net-canvas" width="340" height="250"></canvas>
<div id="log-box" aria-live="polite"></div>

/* Styles for the NEAT demo panel and canvas */
* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; margin: 0; color: #222; background: #fff; }
.panel { display: flex; flex-direction: column; gap: .4rem; padding: .5rem 0; }
.controls { display: flex; align-items: center; gap: .5rem; flex-wrap: wrap; }
button { font: 600 13px system-ui; padding: .4rem .85rem; border: 1px solid #1d3557;
         background: #1d3557; color: #fff; border-radius: 8px; cursor: pointer; }
button.ghost { background: #fff; color: #1d3557; }
.gen-label { font-size: .9rem; color: #555; margin-left: .25rem; }
.status-row { display: flex; gap: .5rem; flex-wrap: wrap; }
.badge { font-size: .8rem; background: #e8eef3; border: 1px solid #cdd9e3; border-radius: 6px;
         padding: .2rem .55rem; color: #333; }
#net-canvas { border: 1px solid #dde3ea; border-radius: 10px; display: block;
              max-width: 100%; background: #f7f9fb; }
#log-box { font-size: .78rem; color: #555; min-height: 2.2em; margin-top: .35rem;
           padding: .2rem .1rem; line-height: 1.5; }

/* NEAT-style neuroevolution for a 2-input XOR task */
const i18nFmt = (s, v) => s.replace(/%(\w+)%/g, (_, k) => String(v[k] ?? ''));

/* XOR training data — 2 inputs, 1 output */
const XOR_IN = [[0,0],[0,1],[1,0],[1,1]];
const XOR_OUT = [0, 1, 1, 0];

/* Global innovation counter — ensures unique historical markings */
let innovCounter = 0;
function newInnov() { return ++innovCounter; }

function sigmoid(x) { return 1 / (1 + Math.exp(-4.9 * x)); }

/* Create a minimal genome — inputs connected directly to output */
function createGenome(numIn, numOut) {
  const nodes = [];
  for (let i = 0; i < numIn; i++) nodes.push({ id: i, type: 'input' });
  for (let i = 0; i < numOut; i++) nodes.push({ id: numIn + i, type: 'output' });
  const conns = [];
  for (let i = 0; i < numIn; i++) {
    for (let j = 0; j < numOut; j++) {
      conns.push({ from: i, to: numIn + j, weight: randW(), enabled: true, innov: newInnov() });
    }
  }
  return { nodes, conns, fitness: 0 };
}

function randW() { return (Math.random() * 2 - 1) * 2; }

/* Evaluate genome fitness on all 4 XOR patterns (max fitness = 4) */
function evalGenome(g) {
  let err = 0;
  for (let t = 0; t < XOR_IN.length; t++) {
    const out = activate(g, XOR_IN[t]);
    err += (out - XOR_OUT[t]) ** 2;
  }
  g.fitness = Math.max(0, 4 - err);
  return g.fitness;
}

function activate(g, inputs) {
  const vals = {};
  g.nodes.forEach(n => { if (n.type === 'input') vals[n.id] = inputs[n.id]; else vals[n.id] = 0; });
  /* Forward pass — process hidden nodes before output nodes */
  const outputIds = g.nodes.filter(n => n.type === 'output').map(n => n.id);
  const hiddenIds = g.nodes.filter(n => n.type === 'hidden').map(n => n.id);
  const order = [...hiddenIds, ...outputIds];
  for (const nid of order) {
    let sum = 0;
    for (const c of g.conns) {
      if (c.enabled && c.to === nid) sum += (vals[c.from] ?? 0) * c.weight;
    }
    vals[nid] = sigmoid(sum);
  }
  return vals[outputIds[0]];
}

/* Mutation operators — weight perturbation, add connection, add node */
function mutate(g) {
  const clone = JSON.parse(JSON.stringify(g));
  const r = Math.random();
  if (r < 0.70) {
    /* Perturb all connection weights with probability 0.8 each */
    for (const c of clone.conns) {
      if (Math.random() < 0.8) c.weight += (Math.random() * 2 - 1) * 0.5;
    }
  } else if (r < 0.85) {
    /* Add a new connection between two previously unconnected nodes */
    addConnectionMutation(clone);
  } else {
    /* Split an existing connection with a new hidden node */
    addNodeMutation(clone);
  }
  return clone;
}

function addConnectionMutation(g) {
  const ids = g.nodes.map(n => n.id);
  const nonInputIds = g.nodes.filter(n => n.type !== 'input').map(n => n.id);
  for (let attempt = 0; attempt < 10; attempt++) {
    const from = ids[Math.floor(Math.random() * ids.length)];
    const to = nonInputIds[Math.floor(Math.random() * nonInputIds.length)];
    if (from === to) continue;
    if (g.conns.some(c => c.from === from && c.to === to)) continue;
    g.conns.push({ from, to, weight: randW(), enabled: true, innov: newInnov() });
    break;
  }
}

function addNodeMutation(g) {
  const eligible = g.conns.filter(c => c.enabled);
  if (!eligible.length) return;
  const old = eligible[Math.floor(Math.random() * eligible.length)];
  old.enabled = false;
  const newId = Math.max(...g.nodes.map(n => n.id)) + 1;
  g.nodes.push({ id: newId, type: 'hidden' });
  g.conns.push({ from: old.from, to: newId, weight: 1, enabled: true, innov: newInnov() });
  g.conns.push({ from: newId, to: old.to, weight: old.weight, enabled: true, innov: newInnov() });
}

/* Population management — elitism + mutation */
const POP = 20;
let population = [];
let generation = 0;
let bestEver = null;

function initPop() {
  innovCounter = 0;
  generation = 0;
  population = [];
  for (let i = 0; i < POP; i++) {
    const g = createGenome(2, 1);
    evalGenome(g);
    population.push(g);
  }
  bestEver = population.slice().sort((a, b) => b.fitness - a.fitness)[0];
}

function evolveOneGen() {
  population.sort((a, b) => b.fitness - a.fitness);
  const nextPop = [];
  /* Keep the best individual unchanged (elitism) */
  nextPop.push(JSON.parse(JSON.stringify(population[0])));
  while (nextPop.length < POP) {
    const parent = population[Math.floor(Math.random() * Math.ceil(POP * 0.4))];
    const child = mutate(parent);
    evalGenome(child);
    nextPop.push(child);
  }
  population = nextPop;
  population.sort((a, b) => b.fitness - a.fitness);
  generation++;
  if (population[0].fitness > (bestEver ? bestEver.fitness : 0)) bestEver = population[0];
  return population[0];
}

/* Render the network on the canvas */
const canvas = document.getElementById('net-canvas');
const ctx = canvas.getContext('2d');

function layoutGenome(g) {
  const W = canvas.width, H = canvas.height;
  const pos = {};
  const inputs = g.nodes.filter(n => n.type === 'input');
  const outputs = g.nodes.filter(n => n.type === 'output');
  const hidden = g.nodes.filter(n => n.type === 'hidden');
  inputs.forEach((n, i) => { pos[n.id] = { x: 40, y: H * (i + 1) / (inputs.length + 1) }; });
  outputs.forEach((n, i) => { pos[n.id] = { x: W - 40, y: H * (i + 1) / (outputs.length + 1) }; });
  hidden.forEach((n, i) => { pos[n.id] = { x: W / 2 + (i % 3 - 1) * 40, y: H * (Math.floor(i / 3) + 1) / (Math.floor(hidden.length / 3) + 2) }; });
  return pos;
}

function drawNetwork(g) {
  const W = canvas.width, H = canvas.height;
  ctx.clearRect(0, 0, W, H);
  const pos = layoutGenome(g);
  const maxW = Math.max(...g.conns.filter(c => c.enabled).map(c => Math.abs(c.weight)), 1);
  /* Draw connections — thickness encodes weight magnitude */
  for (const c of g.conns) {
    if (!c.enabled) continue;
    const from = pos[c.from], to = pos[c.to];
    if (!from || !to) continue;
    const lw = Math.max(0.5, Math.abs(c.weight) / maxW * 3.5);
    ctx.beginPath();
    ctx.moveTo(from.x, from.y);
    ctx.lineTo(to.x, to.y);
    ctx.strokeStyle = c.weight > 0 ? 'rgba(29,53,87,0.55)' : 'rgba(200,50,50,0.55)';
    ctx.lineWidth = lw;
    ctx.stroke();
  }
  /* Draw nodes — color encodes type */
  const colors = { input: '#1d6fa5', output: '#0a7d33', hidden: '#c47c00' };
  for (const n of g.nodes) {
    const p = pos[n.id];
    if (!p) continue;
    ctx.beginPath();
    ctx.arc(p.x, p.y, 12, 0, 2 * Math.PI);
    ctx.fillStyle = colors[n.type] || '#888';
    ctx.fill();
    ctx.strokeStyle = '#fff';
    ctx.lineWidth = 2;
    ctx.stroke();
    ctx.fillStyle = '#fff';
    ctx.font = '9px system-ui';
    ctx.textAlign = 'center';
    ctx.textBaseline = 'middle';
    ctx.fillText(n.type[0].toUpperCase() + n.id, p.x, p.y);
  }
}

function updateUI(best) {
  document.getElementById('gen-num').textContent = generation;
  document.getElementById('fit-val').textContent = best.fitness.toFixed(3);
  document.getElementById('node-val').textContent = best.nodes.length;
  document.getElementById('conn-val').textContent = best.conns.filter(c => c.enabled).length;
  drawNetwork(best);
}

function logMsg(msg) {
  document.getElementById('log-box').textContent = msg;
}

/* Initialize population and draw starting network */
initPop();
updateUI(population[0]);
logMsg('Press "Evolve" to start. Blue = input, green = output, orange = hidden.');

document.getElementById('btn-evolve').addEventListener('click', () => {
  const best = evolveOneGen();
  updateUI(best);
  if (best.fitness >= 3.9) {
    logMsg(i18nFmt('Solved! Network converged at generation %gen%.', { gen: generation }));
  } else {
    logMsg(i18nFmt('Gen %gen% — fitness %fit%, %nodes% nodes, %conns% active connections.', { gen: generation, fit: best.fitness.toFixed(3), nodes: best.nodes.length, conns: best.conns.filter(c => c.enabled).length }));
  }
});

document.getElementById('btn-reset').addEventListener('click', () => {
  initPop();
  updateUI(population[0]);
  logMsg('Press "Evolve" to start. Blue = input, green = output, orange = hidden.');
});
Result
Console

Notice how the first generations keep the minimal two-layer structure. Only when a topology mutation proves useful — winning its fitness competition inside its species — does the network grow a new node or connection. This is the core insight of NEAT: complexity is earned, not assumed.

The Real Complexity

Evolving weights alone is already a hard optimization problem: the loss surface has saddle points, local minima, and flat plateaus. NEAT adds an outer search over architectures that makes the combined space far larger.

  • The competing conventions problem. Two functionally identical networks can have completely different gene orderings, making crossover produce broken offspring. NEAT sidesteps this with historical markings (innovation numbers): genes that arose from the same mutation event keep the same marker forever, so crossover lines up matching genes correctly across individuals.
  • Speciation as a protection mechanism. New structural mutations are fragile: a fresh node or connection usually hurts fitness immediately before it can be refined. NEAT groups individuals into species by structural similarity and lets each species compete only within itself, giving novel structures time to mature.
  • Search space size. With nn possible nodes the number of distinct directed graphs is 2n(n−1)2^{n(n-1)} — doubly exponential in nn. NEAT is a heuristic, not an exact search; it never proves it found the best topology, only that it found a working one.
  • Comparison with fixed-architecture methods. Standard neural network training optimises θ∈Rp\theta \in \mathbb{R}^{p} for a fixed graph; NEAT jointly optimises the graph and its weights. Neural architecture search (NAS) attacks the same joint problem but typically decouples the two phases. NEAT interleaves them through evolution.

NEAT is not the fastest path to the best network on a large task, but it is one of the most principled ways to let structure and function co-evolve from first principles.

Where It Matters

NEAT's idea — evolve the wiring, not just the weights — has spread well beyond its original paper:

  • Game-playing agents. The original Stanley & Miikkulainen paper showed NEAT mastering the double-pole balancing benchmark. Later work applied it to Atari games and real-time strategy, where compact hand-designed controllers outperform gradient-trained alternatives on sample efficiency.
  • Robotics and locomotion. Evolving controllers for legged robots is hard with fixed architectures because the right structure depends on the body morphology. NEAT-style methods co-evolve body and brain simultaneously in some systems.
  • HyperNEAT and indirect encoding. An extension called HyperNEAT encodes network connectivity as a geometric pattern (a compositional pattern-producing network, CPPN), allowing the same evolutionary search to scale to millions of connections by exploiting spatial regularity.
  • Open-ended evolution research. NEAT's speciation mechanism is a model for studying how biological diversity arises — related work explores novelty search and quality-diversity algorithms that reward exploration rather than pure fitness.
  • Neural architecture search. Modern NAS methods such as DARTS and evolutionary NAS borrow NEAT's core insight that architecture and weight optimisation should be connected, even if the mechanisms differ.

Whenever the right network shape is not known in advance, NEAT's principle of complexification from minimal starts remains one of the cleanest answers.

Conclusion

NEAT showed that the architecture of a neural network is not something to be hand-designed and then fixed — it is itself a dimension of the search space, one that evolution can navigate. By starting minimal and growing structure only when it earns its fitness, NEAT avoids the curse of designing the right blueprint upfront.

The three mechanisms — innovation numbers, speciation, and complexification — solve specific, concrete problems that would otherwise derail any joint topology-weight search. They are elegant engineering, and they inspired an entire family of neuroevolution algorithms that continue to influence modern neural architecture search and open-ended learning systems.

Next time you see a deep learning model designed by hand, ask: what if we had let evolution choose the wiring?

Share this article

Pick a channel — or use your device's native share sheet.

Comments

Loading comments...

https://www.kipuhub.com/en/article/neat-neuroevolution/Content licensed under CC BY-NC 4.0.