JIT Compilation

Every program you run on a managed runtime — JavaScript in the browser, Python with PyPy, Java on the JVM — starts life as bytecode or source that is interpreted: the runtime reads each instruction and acts on it. Interpreters are flexible and portable, but they pay a cost per instruction that adds up fast in tight loops.

Just-in-time (JIT) compilation is the runtime's answer: instead of deciding everything at compile time, it watches the program while it runs, identifies the code that executes most (the "hot paths"), and compiles that code to native machine instructions on the fly. The result is that a JavaScript loop can eventually run at speeds approaching hand-written C — without you changing a line of source.

The trick comes with a catch. A JIT compiler must make assumptions about types and shapes to generate fast code. If those assumptions turn out to be wrong — say, a function that always saw integers suddenly receives a string — the runtime must deoptimize: throw away the compiled code and fall back to the interpreter until it has gathered enough new information to try again.

Understanding JIT means understanding this feedback loop: profile → compile → assume → deoptimize → repeat.

The simulator below models a classic tiered JIT pipeline. A loop runs repeatedly; you control the iteration count and can inject a type surprise mid-run.

<!-- {{c_html_intro}} -->
<div class="jit-demo">
  <div class="controls">
    <label for="iters">{{label_iters}}: <strong id="iters-val">200</strong></label>
    <input id="iters" type="range" min="50" max="500" value="200" step="50">
    <button id="run-btn" type="button">{{btn_run}}</button>
    <button id="deopt-btn" type="button" class="ghost">{{btn_inject}}</button>
    <button id="reset-btn" type="button" class="ghost">{{btn_reset}}</button>
  </div>
  <div id="chart" class="chart" aria-label="{{chart_aria}}"></div>
  <div id="status" class="status"></div>
  <div class="legend">
    <span class="dot interp"></span> {{legend_interp}}
    <span class="dot baseline"></span> {{legend_baseline}}
    <span class="dot opt"></span> {{legend_opt}}
    <span class="dot deopt"></span> {{legend_deopt}}
  </div>
</div>

/* {{c_css_intro}} */
* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; color: #222; margin: 0; }
.jit-demo { display: flex; flex-direction: column; gap: .6rem; }
.controls { display: flex; flex-wrap: wrap; align-items: center; gap: .5rem; }
label { font-size: .88rem; color: #444; white-space: nowrap; }
input[type=range] { width: 120px; accent-color: #1d6fa5; }
button { font: 600 13px system-ui; padding: .4rem .85rem; border: 1px solid #1d3557;
         background: #1d3557; color: #fff; border-radius: 7px; cursor: pointer; }
button.ghost { background: #fff; color: #1d3557; }
button:disabled { opacity: .4; cursor: default; }
.chart { position: relative; height: 210px; border: 1px solid #d0d8df;
         border-radius: 8px; background: #f9fbfc; overflow: hidden; }
/* tier color bands */
.band { position: absolute; left: 0; right: 0; opacity: .06; }
.bar { position: absolute; bottom: 0; width: 3px; border-radius: 2px 2px 0 0;
       transition: height .05s; }
.bar.interp   { background: #6c757d; }
.bar.baseline { background: #1d6fa5; }
.bar.opt      { background: #0a7d33; }
.bar.deopt    { background: #c92f3c; }
.tier-label { position: absolute; left: 6px; font-size: .7rem; color: #888;
              font-weight: 600; letter-spacing: .04em; }
.status { font-size: .9rem; font-weight: 600; min-height: 1.3em; color: #1d3557; }
.status.ok    { color: #0a7d33; }
.status.deopt { color: #c92f3c; }
.legend { display: flex; flex-wrap: wrap; gap: .5rem .9rem; font-size: .8rem; color: #555; }
.dot { display: inline-block; width: 10px; height: 10px; border-radius: 2px; margin-right: 3px; }
.dot.interp   { background: #6c757d; }
.dot.baseline { background: #1d6fa5; }
.dot.opt      { background: #0a7d33; }
.dot.deopt    { background: #c92f3c; }

// Code not found

Notice the pattern: Interpreter is slow and predictable. After enough iterations the JIT compiler kicks in and throughput jumps dramatically. Inject a type change and watch the runtime deoptimize — throughput drops back to interpreter speed while the JIT relearns the new pattern, then climbs again. This warmup-compile-deoptimize cycle is exactly what V8, HotSpot, and SpiderMonkey do millions of times per second in real programs.

Modern JIT compilers do not jump straight from interpreter to fully optimized code — that would be too expensive for code that only runs a handful of times. Instead they use multiple tiers:

1. Interpreter — starts immediately, zero compilation cost, slow execution. Collects type feedback.
2. Baseline (warm) JIT — compiles quickly with minimal optimization after a few hundred calls. Fast enough for most code.
3. Optimizing (hot) JIT — full speculative optimization after thousands of iterations. Assumes types stay stable, inlines call targets, eliminates bounds checks. Can be $10\times$ to $100\times$ faster than the interpreter.

Speculative optimization is the key idea. The JIT looks at what types actually appeared in a function and generates code that only works for those types — trusting that the future will resemble the past. When it doesn't:

• The runtime detects the type mismatch at a guard check baked into the compiled code.
• It triggers deoptimization: the stack frame is reconstructed in interpreted form, the compiled version is discarded, and execution continues in the interpreter.
• After more profiling the JIT may recompile, this time with a broader type assumption or a type-check branch built in.

The cost of deoptimization is real but bounded: a deopt event is expensive (microseconds) but rare in well-typed code. Code that constantly changes types — called megamorphic in V8 parlance — may never stabilize and remain in a slower tier permanently.

This is closely related to how program equivalence works: the JIT must prove (speculatively) that its optimized version behaves identically to the original for the observed types.

JIT compilation is the invisible performance layer behind much of modern software:

• JavaScript engines: V8 (Chrome, Node.js), SpiderMonkey (Firefox) and JavaScriptCore (Safari) all use multi-tier JITs. Without them, the web apps you use daily would be orders of magnitude slower.
• The Java Virtual Machine: HotSpot's C1/C2 tiered compiler turned Java from a slow interpreted language in 1995 into one of the fastest general-purpose runtimes available. Android's ART does a hybrid of JIT and ahead-of-time compilation.
• PyPy: a Python implementation that replaces CPython's pure interpreter with a tracing JIT, achieving $5\times$ to $10\times$ speedups on numeric code with zero source changes.
• .NET CLR: the Common Language Runtime JIT-compiles CIL (Common Intermediate Language) bytecode to native code, enabling C#, F# and VB.NET to share a single fast runtime.
• Database query engines: systems like DuckDB and Apache Spark generate machine code (via LLVM) for each query's execution plan at runtime, turning SQL into loops that run as fast as hand-written C.
• WebAssembly: browsers JIT-compile Wasm bytecode to native code, bringing near-native performance to the web for compute-heavy workloads like video codecs and physics engines.

The underlying tension — flexibility of a high-level language, speed of native code — is the same challenge studied in program synthesis and compiler optimization broadly.

JIT compilation is a beautiful feedback loop: the runtime watches your program, bets on the types it sees, compiles a fast version of the hot code, and quietly rolls back that bet whenever reality surprises it. The net effect is that a high-level scripting language can reach speeds that would have seemed impossible without a JIT.

The lesson generalizes: the best time to optimize is when you have data. Ahead-of-time compilers must be conservative because they can't see runtime values; a JIT gets to cheat by looking at what actually happened. That cheat is revokable — deoptimization is the price — but in practice it pays off enormously.

Next time a browser benchmark surprises you with its speed, remember: the engine has been quietly rewriting itself behind the scenes, one hot loop at a time.