Floating Point (IEEE 754)

Every programmer sooner or later types 0.1 + 0.2 into a REPL and gets back 0.30000000000000004. It looks like a bug, but it is the deliberate behavior of IEEE 754, the standard that governs floating-point arithmetic on virtually every chip made since 1985.

The problem it solves is fundamental: real numbers are infinite in count and can have infinite decimal expansions, but a computer register is finite — just 64 bits for a double-precision float. IEEE 754 chooses a clever encoding that covers an enormous range (from roughly 5 × $10^{-324}$ to 1.8 × $10^{308}$) by sacrificing exactness for most values.

The result is a system where 1.0, 2.0 and 0.5 are represented perfectly, but 0.1 is not — because 0.1 in binary is a repeating fraction, just as 1/3 is in decimal. The standard was designed and championed by William Kahan (University of California, Berkeley), who received the Turing Award in 1989 largely for this work. It has been proven optimal in the sense that no other fixed-width binary format achieves the same precision guarantees.

Understanding IEEE 754 is not just trivia. It is the first step toward numerical stability — knowing when to trust a floating-point result and when to rewrite your algorithm to avoid catastrophic cancellation.

Type any decimal number below and watch its IEEE 754 double-precision encoding appear in real time. The 64 bits break into three fields: 1 sign bit, 11 exponent bits, and 52 mantissa bits. Then click Add 0.1 + 0.2 to see both addends side by side and confirm the result is not exactly 0.3.

<div class="controls">
  <label for="num-input">{{label_enter_num}}</label>
  <input id="num-input" type="text" value="0.1" autocomplete="off" spellcheck="false" />
  <button id="btn-add" type="button">{{btn_add}}</button>
  <button id="btn-reset" type="button" class="ghost">{{btn_reset}}</button>
</div>
<div id="bit-display" class="bit-display">
  <div class="bit-row" id="row-a"></div>
</div>
<div id="info" class="info-box"></div>
<div id="add-section" class="add-section hidden">
  <h3>{{heading_add}}</h3>
  <div class="addend-label">0.1</div>
  <div class="bit-row" id="row-01"></div>
  <div class="addend-label">0.2</div>
  <div class="bit-row" id="row-02"></div>
  <div class="sum-label">{{sum_label}} <span id="sum-val"></span></div>
  <div class="addend-label">{{nearest_double}}</div>
  <div class="bit-row" id="row-03"></div>
  <div id="diff-note" class="diff-note"></div>
</div>

* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; margin: 0; color: #1a1a2e; font-size: 14px; }
.controls { display: flex; flex-wrap: wrap; gap: .5rem; align-items: center; margin-bottom: .8rem; }
label { font-weight: 600; }
input[type=text] { font: 600 15px ui-monospace, monospace; padding: .35rem .6rem;
  border: 1.5px solid #6b7eb8; border-radius: 6px; width: 140px; color: #1a1a2e; }
button { font: 600 13px system-ui; padding: .38rem .8rem; border-radius: 6px; cursor: pointer;
  background: #1d3557; color: #fff; border: 1.5px solid #1d3557; }
button.ghost { background: #fff; color: #1d3557; }
button:hover { opacity: .85; }
.bit-display { margin: .5rem 0; }
.bit-row { display: flex; gap: 2px; flex-wrap: wrap; align-items: center; }
.bit { width: 18px; height: 28px; display: flex; align-items: center; justify-content: center;
  font: 700 12px ui-monospace, monospace; border-radius: 4px; border: 1px solid transparent;
  transition: background .15s; }
.bit.sign { background: #ffd6d6; border-color: #e63946; color: #c0392b; }
.bit.exp { background: #d6e8ff; border-color: #457bb8; color: #1a3a6b; }
.bit.mant { background: #d6ffe8; border-color: #28a745; color: #155724; }
.sep { width: 6px; }
.info-box { font-size: .88rem; line-height: 1.6; margin: .5rem 0 .8rem;
  background: #f0f4ff; border-left: 3px solid #1d3557; padding: .5rem .8rem; border-radius: 0 6px 6px 0; }
.info-box .exact { color: #1a6e32; font-weight: 600; }
.info-box .approx { color: #c0392b; font-weight: 600; }
.legend { display: flex; gap: .8rem; flex-wrap: wrap; margin-bottom: .4rem; font-size: .82rem; }
.leg { display: flex; align-items: center; gap: 4px; }
.leg-dot { width: 14px; height: 14px; border-radius: 3px; }
.add-section { margin-top: .8rem; }
.add-section h3 { font-size: .95rem; margin: 0 0 .4rem; color: #1d3557; }
.addend-label { font: 600 12px ui-monospace, monospace; color: #555; margin: .3rem 0 .1rem; }
.sum-label { font: 600 14px ui-monospace, monospace; margin: .4rem 0 .2rem; color: #1d3557; }
.diff-note { font-size: .85rem; color: #c0392b; margin-top: .3rem; line-height: 1.5; }
.hidden { display: none; }

// Code not found

Notice that 0.1 and 0.2 already carry tiny rounding errors before they are added. When the CPU sums them the errors compound, and the nearest representable double is 0.30000000000000004 — not 0.3. The math is exact given the inputs; the inputs are the approximations.

IEEE 754 is not an open problem — it is a solved engineering standard, ratified in 1985 and revised in 2008. Its correctness results are proven: every basic operation (+, -, $\times$, /, $\surd$) is guaranteed to return the closest representable double to the true mathematical result. This property is called correct rounding.

So where does the complexity live?

• Representation gaps: not every decimal maps to a binary float. 0.1 rounds to 0.1000000000000000055511151231257827021181583404541015625 — close, but not equal. These gaps are unavoidable; there are uncountably many reals and only $2^{64}$ doubles.
• Catastrophic cancellation: subtracting two nearly equal numbers destroys significant digits. $(1 + 1e-15) - 1$ should be 1e-15; in double precision it returns 1.1102230246251565e-15, off by 11%. Algorithms that trigger this need to be restructured.
• Non-associativity: $(a + b) + c$ is not always equal to $a + (b + c)$ in floating point. Compilers that reorder floating-point operations for speed can silently change your answer.
• The transcendentals: sin, cos, $\log$ and friends are not required by IEEE 754 to be correctly rounded in the 2008 standard (only the basic ops are). Getting faithfully-rounded transcendentals is a research problem; the CRlibm project at INRIA solved it for double precision.

The complexity status: not a hardness result, but a rich landscape of precision pitfalls that every numerical algorithm must navigate. Related reading: P vs NP for the question of what problems are tractable; fast-multiplication for how arithmetic itself can be made asymptotically faster.

IEEE 754 underpins almost every computed number on the planet. Its trade-offs show up in surprising places:

• Finance: currencies must not be stored in floats. $0.10 has no exact double representation, so summing thousands of them accumulates error. The fix is integer arithmetic in the smallest denomination (cents, satoshis) or a decimal-arithmetic library (Python's decimal, Java's BigDecimal).
• Machine learning: the GPU revolution in deep learning runs on 32-bit (float32) and increasingly 16-bit (float16 / bfloat16) arithmetic — deliberately lower precision to pack more numbers per memory fetch. Gradient underflow and overflow are real training hazards managed with "loss scaling."
• Scientific computing: climate models, orbital mechanics and finite-element simulations run for thousands of hours of CPU time. A single catastrophic cancellation in an inner loop can corrupt a result imperceptibly — numerical analysts spend careers designing algorithms that stay stable under IEEE 754.
• Compilers and reproducibility: the --ffast-math flag on GCC/Clang allows non-IEEE-compliant optimizations for speed. Code compiled with it can produce different answers on different machines or after an innocuous refactor, making reproducibility a nightmare.
• Testing and debugging: NaN != NaN is true by IEEE definition — the only value not equal to itself. Code that tests if (x == x) to detect NaN is a known idiom. Infinity, -0.0 and subnormals add more special-case traps that unit tests regularly miss.

IEEE 754 is not a bug waiting to be fixed — it is a carefully negotiated deal. You get an enormous range, correct rounding on every basic operation, and hardware so fast it barely registers in a benchmark. In exchange, you give up the ability to represent most decimal fractions exactly.

0.1 + 0.2 = 0.30000000000000004 is not sloppy engineering. It is the only answer a 64-bit machine can give while honoring the rest of the contract. William Kahan's design has been so successful that every desktop, phone, and data-center chip implements it — a 1985 standard still running the arithmetic of the modern world.

The lesson is not to distrust floating point, but to understand when the approximation matters. Count money in integers. Test floating-point results with tolerances, not equality. Know when to reach for arbitrary-precision libraries. And the next time 0.1 + 0.2 surprises a junior colleague, you'll know exactly which 64 bits to blame.