Pipelining Hazards

Modern processors do not execute one instruction at a time. They overlap them in an assembly line — a pipeline — so that while one instruction is being decoded the previous one is already being executed and the one before that is writing its result back to a register. On a five-stage RISC pipeline the stages are Fetch → Decode → Execute → Memory → Write-back.

This overlap is why a 3 GHz chip can retire billions of operations per second. But the assembly-line metaphor has a catch: sometimes one worker needs the output of the worker two steps behind them, or the whole line needs to change direction mid-stream. These collisions are called hazards, and they come in three flavours:

• Data hazards — an instruction needs a value that has not been written yet by an earlier instruction still in the pipe. The classic case is a RAW (Read-After-Write) dependency: instruction $i+1$ tries to read a register that instruction $i$ is still computing.
• Control hazards — a branch instruction changes the program counter, but the next one or two instructions after it have already been fetched. Those instructions may be wrong.
• Structural hazards — two instructions need the same hardware resource (say, a single memory port) at the same cycle.

Without any countermeasures the pipeline must insert idle cycles — bubbles (also called stalls or NOPs) — to let the dependency resolve. Forwarding (also called bypassing) short-circuits those waits by routing a result directly from one pipeline stage to the input of an earlier stage. The gain is striking: a RAW that costs two stall cycles with no forwarding costs zero stall cycles with forwarding.

The demo below shows a two-instruction sequence with a classic RAW data hazard. Instruction 1 computes a value; Instruction 2 immediately reads that register.

<!-- {{c_html_comment}} -->
<div class="controls">
  <label class="toggle-label">
    <input type="checkbox" id="fwd-toggle">
    <span class="toggle-track"><span class="toggle-thumb"></span></span>
    <span class="toggle-text">{{lbl_forwarding}}</span>
  </label>
  <button id="btn-step" type="button">{{btn_step}}</button>
  <button id="btn-reset" type="button" class="ghost">{{btn_reset}}</button>
</div>
<div id="pipeline-grid" class="pipeline-grid"></div>
<div id="info-box" class="info-box">{{msg_start}}</div>

/* {{c_css_comment}} */
* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; margin: 0; color: #222; }
.controls { display: flex; align-items: center; gap: .6rem; flex-wrap: wrap; margin-bottom: .8rem; }
.toggle-label { display: flex; align-items: center; gap: .45rem; cursor: pointer; user-select: none; font-size: .9rem; }
.toggle-track { position: relative; width: 36px; height: 20px; background: #cdd; border-radius: 10px; transition: background .2s; flex-shrink: 0; }
.toggle-thumb { position: absolute; top: 3px; left: 3px; width: 14px; height: 14px; background: #fff; border-radius: 50%; transition: left .2s; }
#fwd-toggle:checked + .toggle-track { background: #1d3557; }
#fwd-toggle:checked + .toggle-track .toggle-thumb { left: 19px; }
#fwd-toggle { display: none; }
button { font: 600 13px system-ui; padding: .4rem .85rem; border: 1px solid #1d3557; background: #1d3557; color: #fff; border-radius: 8px; cursor: pointer; }
button.ghost { background: #fff; color: #1d3557; }
.pipeline-grid { display: grid; grid-template-columns: 100px repeat(8, 1fr); gap: 3px; }
.gh { /* {{c_gh}} */ font-size: .72rem; font-weight: 700; color: #555; text-align: center; padding: 3px 0; }
.row-label { font-size: .8rem; font-weight: 600; color: #333; display: flex; align-items: center; padding: 0 4px; }
.cell { height: 36px; border-radius: 6px; display: flex; align-items: center; justify-content: center; font-size: .75rem; font-weight: 700; color: #fff; text-align: center; }
.cell-if  { background: #4361ee; }
.cell-id  { background: #3a86ff; }
.cell-ex  { background: #2a9d8f; }
.cell-mem { background: #e9c46a; color: #333; }
.cell-wb  { background: #f4a261; color: #333; }
.cell-bub { background: #c0c5ce; color: #555; }
.cell-empty { background: transparent; }
.info-box { margin-top: .7rem; font-size: .9rem; line-height: 1.5; padding: .5rem .7rem; background: #f0f4f8; border-left: 3px solid #1d3557; border-radius: 4px; min-height: 2.5em; }
.info-box.ok  { border-color: #2a9d8f; background: #e8f8f5; }
.info-box.bad { border-color: #e76f51; background: #fdf0ec; }
.info-box.done { border-color: #4361ee; background: #eef0fd; }
.legend { display: flex; gap: .5rem; flex-wrap: wrap; margin-top: .6rem; font-size: .72rem; }
.leg-item { display: flex; align-items: center; gap: .25rem; }
.leg-swatch { width: 12px; height: 12px; border-radius: 3px; flex-shrink: 0; }

// Code not found

With forwarding off, the pipeline must wait: a bubble is injected so that Instruction 2 reaches the Execute stage only after Instruction 1 has written its result. With forwarding on, the result is routed directly from the Execute output of Instruction 1 to the Execute input of Instruction 2 — the bubble disappears entirely and the pipeline flows uninterrupted.

The toy demo captures the core idea, but real pipelines are far messier.

All three hazard types in depth:

• Data hazards come in three sub-types. A RAW (Read-After-Write, also called true dependency) is the most common: a later instruction reads a register before an earlier one writes it. A WAR (Write-After-Read, anti-dependency) and WAW (Write-After-Write, output dependency) matter mainly in out-of-order and superscalar processors — they are resolved there through register renaming.
• Control hazards arise at every branch. A conditional branch is not resolved until the Execute stage, so the two instructions fetched after it may be from the wrong path. The naive fix is to stall until the branch resolves (branch penalty = 2 cycles on a classic 5-stage pipe). Modern CPUs use branch prediction: they speculatively fetch the predicted path and flush the pipeline if the prediction is wrong. Accuracy above 95 % is common; a misprediction still costs 10–20 cycles on deep pipelines.
• Structural hazards occur when two instructions contend for the same resource simultaneously. Classic examples: a unified instruction/data cache (solved by Harvard architecture — separate caches), or a single multiply unit that takes multiple cycles.

How forwarding works precisely: after the Execute stage of instruction $i$, its result sits in the EX/MEM pipeline register. If instruction $i+1$ needs that value in its Execute stage, the forwarding unit detects the RAW (by comparing register numbers) and muxes the EX/MEM value into the ALU input — no stall required. A second forwarding path routes from the MEM/WB register for two-cycle distances.

Beyond simple stalls: out-of-order processors (pioneered by the Tomasulo algorithm era) reorder instructions dynamically to fill bubbles with independent work. They also use a reorder buffer to commit results in program order while executing out of order — hiding most hazard penalties at the cost of significant hardware complexity.

The story of pipelining hazards is really the story of how computer architects spent decades hiding latency: first with forwarding, then with branch predictors, then with out-of-order execution, then with speculative execution — each layer adding performance and complexity in equal measure.

Pipelining hazards reach far beyond hardware design:

• Compiler scheduling: a good compiler reorders instructions to place independent work in the stall slots left by hazards. This is called instruction scheduling and is one of the classic dynamic programming problems in compiler back-ends — fill the pipeline without changing program semantics.
• ISA design: the MIPS architecture originally required programmers (and compilers) to fill delay slots after branches with useful instructions, making the hazard visible at the ISA level. Modern RISC-V hides it entirely in hardware.
• Branch prediction units: the entire field of hardware branch prediction (two-bit saturating counters, local/global history, TAGE predictors) exists solely to reduce control-hazard penalties. A modern high-performance predictor is one of the most complex combinational circuits in a CPU.
• Security: the Spectre (2018) and Meltdown (2018) vulnerabilities exploit speculative execution — the mechanism CPUs use to hide control-hazard latency. When a mispredicted branch is flushed, its side-effects on the cache are not always cleaned up, leaking information across security boundaries.
• GPU design: GPU pipelines hide hazards very differently — through massive thread-level parallelism. When one thread stalls on a memory access, the GPU switches instantly to another warp, keeping the execution units busy.

Every time you wonder why a function call is slow, why a switch on an unpredictable value costs more than one on a predictable value, or why security patches sometimes halve memory bandwidth, you are looking at the downstream effects of pipelining hazards.

Pipelining gives processors their speed by overlapping work — and hazards are the inevitable price of that overlap. A single RAW dependency between two adjacent instructions would stall the pipeline for two cycles without forwarding; forwarding eliminates that penalty entirely by routing the result before it even reaches a register.

But the story does not end there. Branch mispredictions flush tens of cycles of speculative work. Out-of-order execution hides remaining stalls by reordering instructions on the fly. And the same speculative machinery that makes CPUs fast turned out to be the root cause of an entire class of security vulnerabilities.

Understanding pipelining hazards means understanding that speed and correctness are in constant tension inside every processor — and that the elegant engineering tricks invented to resolve that tension have consequences that reach all the way to your browser's security model.