Memory Interleaving

Every time your CPU asks for a byte of data, memory makes it wait. A modern DRAM chip needs tens of nanoseconds just to open a row and deliver its contents — and while it recovers, nothing else can happen. At gigahertz clock speeds that pause is an eternity.

Memory interleaving is the answer the hardware industry settled on decades ago. Instead of one giant bank, the address space is sliced across $k$ independent banks, each with its own data and control lines. Consecutive addresses land in consecutive banks: address 0 in bank 0, address 1 in bank 1, … address $k-1$ in bank $k-1$, then address $k$ wraps back to bank 0.

The payoff appears the moment you read a sequential burst. While bank 0 is still recovering from the first request, bank 1 already starts serving the second, bank 2 the third, and so on. The banks work in pipeline — and if there are enough of them, by the time bank 0 is asked for its next word it has already recovered. Latency hasn't shrunk at all; throughput has multiplied by $k$.

This is the same insight behind CPU pipelines and instruction-level parallelism: overlap independent work so that idle hardware time disappears. Memory interleaving just applies it to the memory bus.

Choose how many banks you want and how long each bank takes to respond. Hit Run burst to watch a 8-word sequential read complete — either serialized through one bank or pipelined across all of them.

<!-- {{c_html_intro}} -->
<div class="controls">
  <label>{{lbl_banks}}
    <select id="bankCount">
      <option value="1">1</option>
      <option value="2">2</option>
      <option value="4" selected>4</option>
      <option value="8">8</option>
    </select>
  </label>
  <label>{{lbl_latency}}
    <select id="latency">
      <option value="2">2</option>
      <option value="4" selected>4</option>
      <option value="6">6</option>
      <option value="8">8</option>
    </select>
  </label>
  <button id="runBtn" type="button">{{btn_run}}</button>
</div>
<div id="bankViz" class="bank-viz" aria-label="{{aria_banks}}"></div>
<div class="result-area">
  <div class="result-row">
    <span class="label-col">{{lbl_single}}</span>
    <div class="bar-wrap"><div id="barSingle" class="bar bar-single"></div></div>
    <span id="timeSingle" class="time-label">-</span>
  </div>
  <div class="result-row">
    <span class="label-col">{{lbl_interleaved}}</span>
    <div class="bar-wrap"><div id="barInterleaved" class="bar bar-interleaved"></div></div>
    <span id="timeInterleaved" class="time-label">-</span>
  </div>
</div>
<p id="status" class="status"></p>

/* {{c_css_intro}} */
* { box-sizing: border-box; margin: 0; padding: 0; }
body { font-family: system-ui, sans-serif; color: #222; padding: .6rem; }
.controls { display: flex; flex-wrap: wrap; gap: .6rem; align-items: center; margin-bottom: .8rem; }
label { font-size: .85rem; display: flex; align-items: center; gap: .3rem; }
select { font-size: .85rem; padding: .2rem .4rem; border: 1px solid #aaa; border-radius: 6px; }
button { font: 600 14px system-ui; padding: .4rem .9rem; background: #1d3557; color: #fff;
         border: none; border-radius: 8px; cursor: pointer; }
button:disabled { opacity: .5; cursor: default; }
/* {{c_css_banks}} */
.bank-viz { display: flex; gap: 6px; margin-bottom: .8rem; flex-wrap: wrap; }
.bank { border: 1.5px solid #adb1b8; border-radius: 8px; overflow: hidden;
        width: 56px; min-width: 56px; }
.bank-header { font-size: .7rem; font-weight: 700; text-align: center;
               padding: 2px 0; background: #e8eef3; color: #1d3557; }
.bank-cells { display: flex; flex-direction: column; }
.bank-cell { height: 22px; display: flex; align-items: center; justify-content: center;
             font-size: .72rem; font-family: ui-monospace, monospace; color: #444;
             border-top: 1px solid #e0e4e8; transition: background .18s; }
.bank-cell.active { background: #f4a261; color: #222; font-weight: 700; }
.bank-cell.done { background: #52b788; color: #fff; }
/* {{c_css_bars}} */
.result-area { margin-bottom: .6rem; }
.result-row { display: flex; align-items: center; gap: .5rem; margin-bottom: .4rem; }
.label-col { font-size: .78rem; width: 80px; flex-shrink: 0; font-weight: 600; color: #444; }
.bar-wrap { flex: 1; background: #eee; border-radius: 4px; height: 18px; overflow: hidden; }
.bar { height: 100%; width: 0; border-radius: 4px; transition: width .4s ease; }
.bar-single { background: #e63946; }
.bar-interleaved { background: #2a9d8f; }
.time-label { font-size: .8rem; width: 52px; flex-shrink: 0; text-align: right; }
.status { font-size: .88rem; font-weight: 600; min-height: 1.3em; color: #1d3557; }

// Code not found

With a single bank every request must wait for the previous one to finish, so total time grows linearly. With $k$ banks the requests pipeline: the burst completes in roughly $\text{latency} + (n-1) \cdot \lceil \text{latency}/k \rceil$ cycles instead of $n \cdot \text{latency}$. Add more banks and watch the bar for the interleaved case shrink toward the minimum.

Interleaving is elegant, but it has a structural Achilles heel rooted in the address mapping itself.

With $k$ banks, address $a$ maps to bank $a \bmod k$. Sequential access — stride 1 — hits every bank in round-robin order, which is exactly what the design wants. But a stride-$k$ pattern (addresses $0, k, 2k, \ldots$) maps every address to bank 0. All requests queue on the same bank; the other $k-1$ banks sit idle while latency multiplies.

• Best case (stride 1): throughput $\approx k \times$ a single bank, latency effectively hidden.
• Worst case (stride $k$): throughput identical to a single bank — no gain at all.
• General stride $s$: only $\gcd(s, k)^{-1} \cdot k$ distinct banks are touched. Choosing $k$ as a power of two makes strides that are also powers of two catastrophically conflict-prone.

This is why many high-performance systems use a prime number of banks: no non-trivial stride divides a prime, so every regular pattern distributes evenly. The trade-off is that power-of-two address arithmetic becomes slightly messier.

The same tension appears in hash tables and load balancing: a mapping that distributes one access pattern perfectly can be adversarial for another. Good interleaving design — like good hashing — must reason about the distribution of accesses, not just the average case.

The interleaving idea surfaces in almost every layer of the memory hierarchy:

• Multi-channel DRAM: modern CPUs ship with two, four, or eight memory channels. Each channel is an independent bank group; the memory controller interleaves across channels to double or quadruple peak bandwidth.
• GPU memory controllers: graphics cards pair a very wide bus (256–512 bits) with many interleaved GDDR or HBM banks, because rendering needs enormous sequential bandwidth for textures and framebuffers.
• RAID-0 disk striping: data is striped across drives in fixed-size chunks — exactly interleaving, at disk scale. Sequential reads and writes fan out across all drives simultaneously.
• Cache banks: on-chip caches are themselves split into banks so that simultaneous load/store instructions can access different banks without stalling each other.
• Prefetch and cache performance: hardware prefetchers exploit the regularity of stride-1 access to issue requests to future banks before the CPU asks, hiding latency entirely.

Understand interleaving and you understand why doubling memory channels can matter more than buying faster sticks — and why measuring bandwidth with the wrong stride can make a 4-channel system look no better than a 1-channel one.

Memory interleaving is one of the oldest tricks in computer architecture, and it works because it turns an unavoidable wait into overlapping work. Map consecutive addresses to independent banks, and a burst that would take $n \cdot \text{latency}$ on a single bank shrinks toward just one latency plus a little pipeline fill time.

The catch — stride-$k$ conflicts — is a reminder that every clever address mapping has a dual: the access pattern that breaks it. The fix, choosing a prime number of banks, is itself a lesson in load balancing: the best distribution scheme is the one that leaves no regularity for an adversary to exploit.

Next time you see "dual-channel" or "quad-channel" on a memory spec sheet, you are looking at interleaving in production — and next time a benchmark delivers surprisingly low bandwidth, a stride mismatch is the first place to look.