The epoll Reactor

Imagine a waiter who freezes solid the moment a customer orders — unable to take any other order until the kitchen delivers the plate. Scale that to a web server and you have the C10k problem: in the late 1990s, serving ten thousand simultaneous connections required ten thousand threads, each burning megabytes of stack while mostly waiting for the network.

Dan Kegel named the problem in 1999. The answer was already taking shape in operating-system kernels: instead of one thread per connection, use one thread that watches all connections at once and reacts only when a socket is actually ready to send or receive.

Linux answered with epoll (introduced in kernel 2.5.44, 2002). The idea is deceptively simple: register your sockets with the kernel, then call epoll_wait — a single system call that blocks until at least one socket is ready. No spinning, no polling every socket manually. The kernel does the watching; your thread does the work.

This pattern — register interest, wait for readiness, act, repeat — has a name: the Reactor. It is the skeleton inside nginx, Node.js, Redis, and virtually every high-throughput server built in the last two decades.

Below is a simulation of the reactor loop. Adjust the number of connections and watch a single thread dispatch only the ones that become ready each tick — just as epoll_wait works in the real kernel.

<!-- {{c_html_intro}} -->
<div class="controls">
  <label>
    {{lbl_connections}}
    <input type="range" id="connCount" min="10" max="200" value="60" step="10">
    <span id="connVal">60</span>
  </label>
  <label>
    {{lbl_ready_pct}}
    <input type="range" id="readyPct" min="0" max="100" value="20" step="5">
    <span id="pctVal">20%</span>
  </label>
</div>
<div class="legend">
  <span class="dot idle"></span> {{leg_idle}}
  <span class="dot ready"></span> {{leg_ready}}
  <span class="dot active"></span> {{leg_dispatched}}
</div>
<canvas id="canvas" width="460" height="220"></canvas>
<div class="stats" id="stats"></div>
<div class="btns">
  <button id="btnTick" type="button">{{btn_tick}}</button>
  <button id="btnAuto" type="button">{{btn_run}}</button>
  <button id="btnReset" type="button" class="ghost">{{btn_reset}}</button>
</div>

* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; color: #222; margin: 0; }
.controls { display: flex; flex-direction: column; gap: .4rem; margin-bottom: .6rem; font-size: .85rem; }
.controls label { display: flex; align-items: center; gap: .5rem; }
.controls input[type=range] { flex: 1; }
.controls span { min-width: 3.2rem; text-align: right; font-variant-numeric: tabular-nums; }
.legend { font-size: .8rem; display: flex; gap: .8rem; align-items: center; margin-bottom: .4rem; color: #555; }
.dot { display: inline-block; width: 10px; height: 10px; border-radius: 50%; }
.dot.idle { background: #c9ccd1; }
.dot.ready { background: #f4a226; }
.dot.active { background: #1d8a4b; }
canvas { display: block; border: 1px solid #dde3ea; border-radius: 8px; background: #f8fafc; max-width: 100%; }
.stats { font-size: .88rem; font-weight: 600; margin: .5rem 0; min-height: 1.4em; color: #1d3557; }
.btns { display: flex; gap: .5rem; flex-wrap: wrap; }
button { font: 600 14px system-ui, sans-serif; padding: .45rem .9rem; border: 1px solid #1d3557;
         background: #1d3557; color: #fff; border-radius: 8px; cursor: pointer; }
button.ghost { background: #fff; color: #1d3557; }

// Code not found

Notice that the thread processes only the ready connections each tick, not all of them. When connections are sparse or idle, the thread does almost nothing. When they all fire at once, it still finishes in one pass — no extra threads needed. This is why epoll scales to $10^{4}$ or even $10^{5}$ sockets while a one-thread-per-connection model collapses.

The elegance of epoll is algorithmic, not just architectural.

• select and poll (the older interfaces): to find which of $n$ sockets is ready, the kernel scans all $n$ file descriptors on every call — $O(n)$ per wakeup. With 10,000 sockets and most of them idle, that is 10,000 useless checks per tick.
• epoll: the kernel maintains a ready-list internally. epoll_wait returns only the sockets that fired — $O(\text{ready})$ per wakeup. Registering or removing a socket is $O(\log n)$ via a red-black tree, but wakeups are essentially free.
• Memory: select copies a bitmask of all $n$ descriptors into kernel space each call. epoll keeps state persistently in the kernel; you only copy the small list of events that fired.
• Edge-triggered mode: epoll can notify you once per state change (not once per byte available), letting you drain sockets at your own pace without redundant notifications.

The reactor loop itself is $O(\text{ready})$ per iteration. A server handling 50,000 idle connections and 200 active ones burns CPU proportional to 200, not 50,000. That is the C10k solution in one sentence.

The pattern also sidesteps the classic load-balancing nightmare: instead of distributing threads across cores, a single-threaded reactor serializes all I/O and avoids lock contention entirely. Multi-core scaling is achieved by running one reactor per core and sharing nothing — the model used by nginx's worker processes.

Once you see the reactor pattern, you see it everywhere:

• Web servers: nginx uses one epoll reactor per worker process. A single nginx worker routinely holds 10,000–50,000 open connections with minimal CPU overhead.
• JavaScript runtimes: Node.js is essentially a reactor (powered by libuv, which wraps epoll on Linux and kqueue on macOS) plus a call-stack. Every await is a reactor registration under the hood.
• In-memory databases: Redis runs a single-threaded reactor for all client I/O. Because commands complete in microseconds, the single thread never blocks long enough to matter.
• Game networking: multiplayer servers track thousands of player sockets in one loop; the reactor keeps latency low without spawning a thread per player.
• OS-level abstractions: io_uring (Linux 5.1, 2019) pushes the idea further — submissions and completions are queued in shared memory, eliminating even the system-call overhead. The reactor principle remains; the interface evolves.

The alternative — one thread or process per connection — hits a wall at a few hundred connections on typical hardware due to context-switch cost and stack memory. epoll is not an optimization; it is a qualitative change in what is possible.

The epoll reactor is one of those ideas that looks like a systems trick but is really an algorithmic insight: stop paying $O(n)$ to ask "which socket is ready?" and start paying $O(\text{ready})$ to receive the answer directly.

That shift — from polling to notification, from threads to events, from $O(n)$ to $O(\text{ready})$ — is what turned the C10k problem from a crisis into a non-issue. Every await in your JavaScript, every request nginx serves without spawning a process, every Redis command that completes in a microsecond traces back to the same kernel primitive.

Understanding epoll means understanding why load-balancing across threads is sometimes the wrong unit of concurrency, and why the right question is not "how many threads?" but "how do we wait?"