Hybrid Logical Clocks

Distributed systems live on many machines with many clocks, and those clocks are never perfectly in sync. That gap creates two hard problems that pull in opposite directions.

Lamport clocks (1978) guarantee causal consistency: if event $A$ caused event $B$, then $\text{ts}(A) < \text{ts}(B)$. But a Lamport timestamp is just a counter — it drifts arbitrarily far from wall-clock time, so you cannot use it to answer questions like "give me everything that happened in the last five minutes."

Physical clocks (NTP, GPS, TrueTime) stay close to real time. But clock skew means two independent nodes can easily assign the same or even reversed timestamps to causally related events.

Hybrid Logical Clocks (HLC), introduced by Kulkarni, Demirbas, Madeppa, Avva and Bharadwaj in 2014, pack both worlds into a single compact timestamp: a physical component $l$ that tracks the maximum physical time seen so far, and a logical counter $c$ that breaks ties and advances only when the physical component cannot. The result is always within a bounded skew $\varepsilon$ of true time — yet it respects causality just as faithfully as a Lamport clock.

The key invariant: for any two events $e$ and $f$ where $e \to f$ (e happened before f), $\text{hlc}(e) < \text{hlc}(f)$ — and $l \leq \text{pt}(\text{now}) + \varepsilon$ where $\text{pt}$ is physical time.

Three nodes exchange messages while their physical clocks drift independently. Watch how their HLC timestamps stay monotone and causally consistent even when the physical clocks slip backward or forward.

<!-- {{c_html_intro}} -->
<div class="intro-text">{{hint_para}}</div>
<div id="network" class="network">
  <div class="node-col" id="col-A">
    <div class="node-label">{{node_a_label}}</div>
    <div class="clock-row"><span class="clock-badge">{{phys_label}}: <span id="pt-A">0</span></span><span class="clock-badge hlc">HLC: <span id="hlc-A">(0,0)</span></span></div>
    <div class="events" id="events-A"></div>
    <div class="node-btns">
      <button id="local-A" type="button">{{btn_local}}</button>
      <button id="send-AB" type="button">{{btn_send_b}}</button>
      <button id="send-AC" type="button">{{btn_send_c}}</button>
    </div>
  </div>
  <div class="node-col" id="col-B">
    <div class="node-label">{{node_b_label}}</div>
    <div class="clock-row"><span class="clock-badge">{{phys_label}}: <span id="pt-B">0</span></span><span class="clock-badge hlc">HLC: <span id="hlc-B">(0,0)</span></span></div>
    <div class="events" id="events-B"></div>
    <div class="node-btns">
      <button id="local-B" type="button">{{btn_local}}</button>
      <button id="send-BA" type="button">{{btn_send_a}}</button>
      <button id="send-BC" type="button">{{btn_send_c}}</button>
    </div>
  </div>
  <div class="node-col" id="col-C">
    <div class="node-label">{{node_c_label}}</div>
    <div class="clock-row"><span class="clock-badge">{{phys_label}}: <span id="pt-C">0</span></span><span class="clock-badge hlc">HLC: <span id="hlc-C">(0,0)</span></span></div>
    <div class="events" id="events-C"></div>
    <div class="node-btns">
      <button id="local-C" type="button">{{btn_local}}</button>
      <button id="send-CA" type="button">{{btn_send_a}}</button>
      <button id="send-CB" type="button">{{btn_send_b}}</button>
    </div>
  </div>
</div>
<div class="status" id="status">{{status_idle}}</div>
<button id="drift-btn" type="button" class="drift-btn">{{btn_drift}}</button>
<button id="reset-btn" type="button" class="ghost">{{btn_reset}}</button>

/* {{c_css_intro}} */
* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; color: #222; margin: 0; padding: 4px; }
.intro-text { font-size: .85rem; color: #444; margin-bottom: .6rem; line-height: 1.45; }
.network { display: flex; gap: 8px; align-items: flex-start; }
.node-col { flex: 1; background: #f0f4f8; border: 1px solid #cdd9e3; border-radius: 10px; padding: 8px; min-width: 0; }
.node-label { font-weight: 700; font-size: 1rem; text-align: center; margin-bottom: 4px; color: #1d3557; }
.clock-row { display: flex; gap: 4px; flex-wrap: wrap; margin-bottom: 6px; }
.clock-badge { font-size: .72rem; background: #dce8f3; border: 1px solid #b8ccdf; border-radius: 6px; padding: 2px 5px; color: #1d3557; }
.clock-badge.hlc { background: #d4edda; border-color: #9acba8; color: #0a5c2f; font-weight: 600; }
.events { min-height: 80px; max-height: 120px; overflow-y: auto; font-size: .75rem; margin-bottom: 6px; }
.event-entry { padding: 2px 0; border-bottom: 1px solid #e2e8ef; line-height: 1.35; }
.event-entry .etag { font-weight: 700; color: #1d3557; }
.event-entry .hlc-val { color: #0a5c2f; font-weight: 600; }
.event-entry .pt-val { color: #888; }
.event-entry.recv { background: #fff9e6; }
.node-btns { display: flex; flex-direction: column; gap: 4px; }
button { font: 600 12px system-ui, sans-serif; padding: .3rem .6rem; border: 1px solid #1d3557;
         background: #1d3557; color: #fff; border-radius: 7px; cursor: pointer; width: 100%; }
button.ghost { background: #fff; color: #1d3557; }
.drift-btn { background: #e63946; border-color: #c92f3c; margin-top: 6px; width: auto; padding: .35rem .9rem; }
.status { font-size: .85rem; font-weight: 600; margin: 6px 0 4px; min-height: 1.3em; color: #1d3557; }

// Code not found

Notice that whenever a message arrives from a node with a higher physical component, the receiver's $l$ jumps up — and when two events share the same $l$, the counter $c$ makes them strictly ordered. The physical clocks may disagree; the HLC timestamps never contradict causality.

The HLC timestamp is a pair $(l, c)$ where $l$ is the logical physical time (highest physical time seen) and $c$ is an integer counter that strictly orders events sharing the same $l$.

The update rule has three cases:

• Local event (no message): set $l' = \max(l, \text{pt})$. If $l' = l$, increment $c$; otherwise reset $c = 0$.
• Send event: same as local — produce $(l', c')$ and attach it to the message.
• Receive event: given the incoming timestamp $(l_m, c_m)$, set $l' = \max(l, l_m, \text{pt})$. Then: if all three were equal, $c' = \max(c, c_m) + 1$; if $l' = l_m$ only, $c' = c_m + 1$; if $l' = l$ only, $c' = c + 1$; otherwise $c' = 0$.

Why it works:

1. Causal consistency — every send/receive pair advances the timestamp, so the happened-before relation $e \to f$ implies $\text{hlc}(e) < \text{hlc}(f)$.
2. Bounded physical drift — $l$ only increases, and it jumps to $\text{pt}$ whenever $\text{pt}$ overtakes it, so $l \leq \text{pt} + \varepsilon$ where $\varepsilon$ is the maximum clock skew in the system.
3. Compact representation — a 64-bit integer suffices: the top $k$ bits for $l$ (milliseconds since epoch) and the bottom $16$ bits for $c$. The pair fits in the same space as a vanilla NTP timestamp.

The logical counter $c$ stays small in practice — it only grows if many events happen within the same millisecond on the same node, which is rare. Compare this to pure Lamport clocks: $c$ there can grow without bound, whereas in HLC it resets to $0$ every time real time advances.

HLC addresses a real pain point that arises any time you need to order events across data centers:

• Distributed databases: CockroachDB, YugabyteDB and TiDB all use HLC (or a close variant) as the timestamp backbone for multi-version concurrency control. A consistent snapshot at time $T$ simply collects every version with $l \leq T$ — no waiting for a global oracle.
• Causally consistent reads: a client that wrote at $(l, c)$ can send that timestamp to any replica and ask for the latest version at or after $(l, c)$. The replica waits until its HLC reaches that point — bounded by $\varepsilon$ — then answers.
• Event sourcing and audit logs: append-only logs that must be merged from multiple nodes can use HLC to produce a total order that is both human-readable (wall-clock proximity) and logically correct.
• Conflict-free replicated data types (CRDTs): many CRDT implementations rely on timestamps to resolve concurrent updates; HLC gives them a timestamp that is simultaneously physical and causal.
• Debugging distributed systems: because $l$ stays close to real time, log entries from different nodes can be compared by HLC and still line up sensibly with human-readable timestamps — unlike pure Lamport clocks.

The tension between physical time and logical causality is not a quirk of distributed systems — it is fundamental. Real clocks drift; causal order must be maintained. HLC resolves the tension with a two-field timestamp and a three-case update rule that fits in 64 bits.

The elegance is in what it avoids: no global time oracle, no expensive synchronization round-trips, no growing unbounded counters. Just a small increment to the physical timestamp when the world cooperates, and a logical tick when it does not.

The next time your distributed system needs to answer "what happened first?", remember that the answer sits in a pair $(l, c)$ — close enough to real time to be useful, logical enough to be correct.