Adaptive Step-Size Integration

Every time a physicist simulates a satellite orbit, a biologist models a chemical reaction, or an engineer tests a control loop, they are solving a differential equation: given how fast a quantity changes now, trace where it goes. No closed formula exists for most real problems, so we march forward in small time steps, computing an approximate value at each one.

The naive approach picks a fixed step size: the same tiny increment from start to finish. That is wasteful. Most trajectories have long, gentle stretches where almost nothing happens — and short, violent bursts where the state changes rapidly. A fixed step that is small enough to survive the burst spends the vast majority of its budget plodding through the calm.

Adaptive step-size control solves both problems at once. The solver estimates its own local error at each step — the difference between a cheap approximation and a slightly better one. If the error is too big, it shrinks the step and retries. If the error is tiny, it stretches the step and moves faster. The result is a solver that automatically clusters its effort at the sharp turns and coasts through the smooth stretches.

This is the core idea behind workhorses like Runge-Kutta 4(5) (Dormand–Prince), the algorithm that powers SciPy's solve_ivp, MATLAB's ode45, and countless scientific applications. The technique is decades old and fully understood — it is not an open problem but a solved piece of numerical craft worth knowing.

The demo below integrates $y' = 2\sin(2t) + 5\,e^{-50(t-1)^2}$ on $[0, 6]$: a smooth sine wave carrying a sharp Gaussian spike near $t = 1$. A fixed-step Euler solver and a simple adaptive RK4 solver run side by side — both chasing the same curve.

<!-- {{c_html_intro}} -->
<p class="hint">{{hint_para}}</p>
<div class="controls">
  <label>{{label_tol}} <input id="tol" type="range" min="1" max="5" step="1" value="3"></label>
  <span id="tol-display" class="tol-val"></span>
</div>
<canvas id="cv" width="560" height="260"></canvas>
<div class="legend">
  <span class="dot fixed"></span> {{legend_fixed}}
  &nbsp;&nbsp;
  <span class="dot adaptive"></span> {{legend_adaptive}}
</div>
<div class="stats" id="stats"></div>
<div class="btns">
  <button id="run" type="button">{{btn_run}}</button>
  <button id="reset" type="button" class="ghost">{{btn_reset}}</button>
</div>

/* {{c_css_intro}} */
* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; color: #222; margin: 0; }
.hint { font-size: .9rem; color: #444; margin: 0 0 .6rem; line-height: 1.45; }
.controls { display: flex; align-items: center; gap: .7rem; margin-bottom: .5rem; font-size: .9rem; }
input[type=range] { width: 120px; }
.tol-val { font-weight: 700; font-size: .9rem; color: #1d3557; min-width: 4rem; }
canvas { display: block; width: 100%; max-width: 560px; border: 1px solid #cdd9e3; border-radius: 8px; background: #f8fafc; }
.legend { display: flex; align-items: center; gap: .5rem; font-size: .85rem; margin: .4rem 0; flex-wrap: wrap; }
.dot { display: inline-block; width: 10px; height: 10px; border-radius: 50%; }
.dot.fixed { background: #adb1b8; }
.dot.adaptive { background: #e63946; }
.stats { font-size: .9rem; font-weight: 600; min-height: 1.4em; color: #1d3557; margin: .2rem 0; }
.btns { display: flex; gap: .5rem; flex-wrap: wrap; margin-top: .4rem; }
button { font: 600 14px system-ui, sans-serif; padding: .45rem .9rem; border: 1px solid #1d3557;
         background: #1d3557; color: #fff; border-radius: 8px; cursor: pointer; }
button.ghost { background: #fff; color: #1d3557; }

// Code not found

Watch how the adaptive solver's dots cluster around the spike near $t = 1$ and then spread far apart on the smooth sinusoidal tail. The fixed solver uses the same number of steps everywhere — paying the same cost for the easy stretches as for the hard ones. The step-count gap shows the computational savings, and tightening the tolerance reveals how the adaptive solver automatically re-clusters.

The machinery inside an adaptive solver is surprisingly compact.

Two approximations per step. A solver like RK4(5) computes two estimates of the next state: one of order $p$ and one of order $p+1$. Their difference $\delta$ is the local truncation error estimate — the cost of taking this step at this size.

The rescaling rule. If the current step $h$ produces error $\delta$ but the user's tolerance is $\varepsilon$, the optimal next step is approximately:

$$h_{\text{new}} = h \cdot \left(\frac{\varepsilon}{\delta}\right)^{1/(p+1)}$$

This formula follows from the fact that the local error scales as $h^{p+1}$. Shrink $h$ by a factor of two and the error drops by $2^{p+1}$ — roughly 32× for a fifth-order method.

The accept/reject loop. If $\delta > \varepsilon$, the step is rejected and retried with $h_{\text{new}}$. If $\delta \leq \varepsilon$, the step is accepted and the solver advances. A safety factor (typically 0.9) prevents oscillation around the boundary.

Stiffness is the hard case. Some ODEs — like chemical reaction networks with very different time scales — force the step size so small that even an adaptive explicit solver crawls. That is the domain of implicit methods and the separate field of stiff equation solvers, which invert a Jacobian at each step to buy larger steps at the cost of linear algebra.

The bottom line: adaptive step-size control is $O(S)$ in the number of accepted steps $S$, where $S$ is determined automatically by the solution's curvature and the user's tolerance. A factor-of-10 tighter tolerance roughly multiplies $S$ by $10^{1/(p+1)}$ — for a fifth-order method, less than a factor of 2. Tight tolerances are cheaper than they look.

Any simulation that follows a changing quantity through time benefits from step adaptivity:

• Orbital mechanics: a satellite glides smoothly far from Earth but accelerates sharply during a close flyby. An adaptive solver naturally concentrates steps near the periapsis and coasts through the cruise phase, delivering accurate trajectories at a fraction of the cost of a fixed-step integrator.
• Pharmacokinetics: drug concentration in the bloodstream drops fast after injection, then decays slowly. Adaptive solvers let pharmacologists fit models to clinical data without hand-tuning a step size for each drug.
• Circuit simulation (SPICE): digital switching events create near-discontinuities in voltage and current. SPICE-family simulators use adaptive time-stepping to catch each edge accurately while flying through the quiet intervals.
• Climate and weather models: atmospheric chemistry involves species with reaction rates spanning twelve orders of magnitude. Splitting the system and using adaptive solvers for the stiff chemical subsystem is standard practice.
• Robotics and control: real-time simulators for legged robots must track contact events — sudden impacts — accurately. Adaptive solvers with event detection locate the exact moment of contact without sampling blindly.

The idea even migrated into numerical optimization: line-search methods and trust-region methods adaptively resize their "step" toward the minimum using exactly the same intuition — measure local curvature, adjust step size accordingly.

Adaptive step-size integration is a masterclass in computational economy. Instead of committing to a fixed budget per unit of time, the solver listens to the problem: it estimates how hard each local piece is and allocates effort accordingly. Sharp turns get many small steps; smooth stretches get a few large ones.

The rescaling formula $h_{\text{new}} = h \cdot (\varepsilon / \delta)^{1/(p+1)}$ is a closed-loop controller — the algorithm controls its own accuracy in real time. Decades of software engineering have layered dense-output interpolation, event detection, and stiffness detection on top of this simple core, but the heart of it has not changed since the 1960s.

The lesson transfers broadly: wherever a computation has regions of wildly varying difficulty, the right strategy is almost always to measure difficulty locally and adapt. That idea appears in mesh refinement for numerical integration, in variable-rate sampling for signal processing, and in the step-size rules of gradient descent. The pattern recurs because it is simply optimal.