NEAT Neuroevolution

Most machine learning training starts with a fixed blueprint: pick a number of layers, choose how many neurons each layer has, and then adjust the weights until the network performs well. The architecture is decided by the engineer; only the numbers inside it are learned.

NEAT — NeuroEvolution of Augmenting Topologies — takes a different path. Introduced by Kenneth O. Stanley and Risto Miikkulainen in 2002, it treats the structure of a neural network as something that can itself be evolved. A NEAT population starts with the simplest possible networks — inputs wired directly to outputs, nothing in between — and mutations gradually add neurons and connections over generations.

Three ideas make this work without collapsing into chaos:

• Innovation numbers tag every new gene (node or connection) with a global timestamp, so two networks that independently invented the same structure can be recognised and crossed over correctly.
• Speciation groups similar individuals together so that a new, structurally different individual is not immediately out-competed before it has a chance to refine its novel structure.
• Complexification from minimal starts means the search begins in the smallest possible space and grows only when complexity earns its keep.

The result is an algorithm that explores the space of both weights and architectures simultaneously, discovering efficient network topologies the engineer never specified.

The demo below runs a simplified NEAT loop on a tiny XOR-like task. Each node in the network is shown as a circle; connections are lines whose thickness reflects their weight strength. Press Evolve one generation to apply mutations and selection, and watch the topology grow.

<!-- {{c_html_comment}} -->
<div class="panel">
  <div class="controls">
    <button id="btn-evolve" type="button">{{btn_evolve}}</button>
    <button id="btn-reset" type="button" class="ghost">{{btn_reset}}</button>
    <span class="gen-label">{{label_gen}} <strong id="gen-num">0</strong></span>
  </div>
  <div class="status-row">
    <span class="badge" id="badge-fitness">{{label_fitness}} <strong id="fit-val">0.00</strong></span>
    <span class="badge" id="badge-nodes">{{label_nodes}} <strong id="node-val">0</strong></span>
    <span class="badge" id="badge-conns">{{label_conns}} <strong id="conn-val">0</strong></span>
  </div>
</div>
<canvas id="net-canvas" width="340" height="250"></canvas>
<div id="log-box" aria-live="polite"></div>

/* {{c_css_comment}} */
* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; margin: 0; color: #222; background: #fff; }
.panel { display: flex; flex-direction: column; gap: .4rem; padding: .5rem 0; }
.controls { display: flex; align-items: center; gap: .5rem; flex-wrap: wrap; }
button { font: 600 13px system-ui; padding: .4rem .85rem; border: 1px solid #1d3557;
         background: #1d3557; color: #fff; border-radius: 8px; cursor: pointer; }
button.ghost { background: #fff; color: #1d3557; }
.gen-label { font-size: .9rem; color: #555; margin-left: .25rem; }
.status-row { display: flex; gap: .5rem; flex-wrap: wrap; }
.badge { font-size: .8rem; background: #e8eef3; border: 1px solid #cdd9e3; border-radius: 6px;
         padding: .2rem .55rem; color: #333; }
#net-canvas { border: 1px solid #dde3ea; border-radius: 10px; display: block;
              max-width: 100%; background: #f7f9fb; }
#log-box { font-size: .78rem; color: #555; min-height: 2.2em; margin-top: .35rem;
           padding: .2rem .1rem; line-height: 1.5; }

// Code not found

Notice how the first generations keep the minimal two-layer structure. Only when a topology mutation proves useful — winning its fitness competition inside its species — does the network grow a new node or connection. This is the core insight of NEAT: complexity is earned, not assumed.

Evolving weights alone is already a hard optimization problem: the loss surface has saddle points, local minima, and flat plateaus. NEAT adds an outer search over architectures that makes the combined space far larger.

• The competing conventions problem. Two functionally identical networks can have completely different gene orderings, making crossover produce broken offspring. NEAT sidesteps this with historical markings (innovation numbers): genes that arose from the same mutation event keep the same marker forever, so crossover lines up matching genes correctly across individuals.
• Speciation as a protection mechanism. New structural mutations are fragile: a fresh node or connection usually hurts fitness immediately before it can be refined. NEAT groups individuals into species by structural similarity and lets each species compete only within itself, giving novel structures time to mature.
• Search space size. With $n$ possible nodes the number of distinct directed graphs is $2^{n(n-1)}$ — doubly exponential in $n$. NEAT is a heuristic, not an exact search; it never proves it found the best topology, only that it found a working one.
• Comparison with fixed-architecture methods. Standard neural network training optimises $\theta \in \mathbb{R}^{p}$ for a fixed graph; NEAT jointly optimises the graph and its weights. Neural architecture search (NAS) attacks the same joint problem but typically decouples the two phases. NEAT interleaves them through evolution.

NEAT is not the fastest path to the best network on a large task, but it is one of the most principled ways to let structure and function co-evolve from first principles.

NEAT's idea — evolve the wiring, not just the weights — has spread well beyond its original paper:

• Game-playing agents. The original Stanley & Miikkulainen paper showed NEAT mastering the double-pole balancing benchmark. Later work applied it to Atari games and real-time strategy, where compact hand-designed controllers outperform gradient-trained alternatives on sample efficiency.
• Robotics and locomotion. Evolving controllers for legged robots is hard with fixed architectures because the right structure depends on the body morphology. NEAT-style methods co-evolve body and brain simultaneously in some systems.
• HyperNEAT and indirect encoding. An extension called HyperNEAT encodes network connectivity as a geometric pattern (a compositional pattern-producing network, CPPN), allowing the same evolutionary search to scale to millions of connections by exploiting spatial regularity.
• Open-ended evolution research. NEAT's speciation mechanism is a model for studying how biological diversity arises — related work explores novelty search and quality-diversity algorithms that reward exploration rather than pure fitness.
• Neural architecture search. Modern NAS methods such as DARTS and evolutionary NAS borrow NEAT's core insight that architecture and weight optimisation should be connected, even if the mechanisms differ.

Whenever the right network shape is not known in advance, NEAT's principle of complexification from minimal starts remains one of the cleanest answers.

NEAT showed that the architecture of a neural network is not something to be hand-designed and then fixed — it is itself a dimension of the search space, one that evolution can navigate. By starting minimal and growing structure only when it earns its fitness, NEAT avoids the curse of designing the right blueprint upfront.

The three mechanisms — innovation numbers, speciation, and complexification — solve specific, concrete problems that would otherwise derail any joint topology-weight search. They are elegant engineering, and they inspired an entire family of neuroevolution algorithms that continue to influence modern neural architecture search and open-ended learning systems.

Next time you see a deep learning model designed by hand, ask: what if we had let evolution choose the wiring?