Hybrid Logical Clocks Lab | mysimulator.uk

Why Neither Physical Nor Logical Clocks Are Enough

Physical clocks on separate machines never agree exactly. Even with network time synchronization protocols, small offsets and drift are unavoidable, so two machines might disagree by a few milliseconds or more at any given moment. If a system simply timestamps every event with the local physical clock and assumes those timestamps can be compared to determine order, it will sometimes get the order wrong: an event that causally happened before another can end up with a larger physical timestamp simply because its machine's clock ran slightly ahead. This is a correctness hazard, not a cosmetic one, since databases and distributed logs often rely on timestamp order to decide which write wins or which read is fresh. Logical clocks, introduced by Lamport, solve the ordering problem by discarding physical time entirely. Each node keeps a counter that increases with every local event and, when a message is received, jumps to one more than the maximum of the local counter and the counter carried in the message. This guarantees that causal order is always reflected correctly: if event A happened before event B, A's Lamport counter is guaranteed to be smaller than B's. The catch is that the counter is a pure abstraction. It has no relationship to real elapsed time, so a node that receives many messages in quick succession can see its counter leap far ahead of what wall-clock time would suggest, and a node that participates in few interactions can lag far behind. Comparing a Lamport timestamp to an actual moment in time, for example to decide whether a record is more than five minutes old, is meaningless. Hybrid Logical Clocks exist precisely to close this gap. They keep timestamps anchored to physical time under normal conditions, so in the common case where clocks are reasonably synchronized and events are not densely concurrent, an HLC timestamp is simply the local wall-clock reading. Only when physical time fails to provide enough resolution, or when causality demands it, does the logical component step in and increment, exactly mirroring the situations where a pure physical clock alone would give an incorrect ordering.

The HLC Timestamp Structure

An HLC timestamp is a pair of values, often written as (physical time, logical counter), sometimes abbreviated as (pt, l). The physical time component is drawn from the node's local wall clock and represents the algorithm's best current estimate of real time, adjusted upward whenever causality requires it to stay ahead of timestamps it has observed. The logical counter component is a small integer that resolves ties and captures ordering information that cannot be expressed by physical time alone, for instance when multiple events occur within the same clock tick or when a received message's timestamp does not exceed the local physical clock. Comparing two HLC timestamps is done lexicographically: first compare the physical time components, and only if those are equal do you compare the logical counters. This means an HLC timestamp behaves almost exactly like a physical timestamp for the purpose of human interpretation and range queries, since the physical component dominates the comparison in the overwhelming majority of cases. The logical counter typically stays small, often single digits, because it only accumulates when several causally related events are packed into the same physical time tick, and it resets to zero whenever the physical clock component genuinely advances past its previous value. This design gives HLC two properties simultaneously that neither pure physical nor pure logical clocks offer alone. First, the physical component is always close to true wall-clock time, bounded by the maximum clock skew and message delay observed in the system, so an HLC timestamp can be used almost interchangeably with a regular timestamp for expiry checks, garbage collection windows, or human-readable logs. Second, the pair as a whole preserves the causal ordering guarantee: whenever one event causally happened before another, comparing their HLC pairs lexicographically will always correctly report that ordering, exactly like a Lamport clock would.

The Update Rule, Step by Step

The HLC algorithm defines two update rules, one for purely local events and one for receiving a message that carries a remote HLC timestamp. Both rules follow the same underlying idea: never let the timestamp go backward, and always incorporate the most recent information available. On a local event, a node reads its physical clock, call the reading pt_now. It then computes the new physical time component as the maximum of pt_now and the node's own previous physical time component. If that maximum equals the previous physical time component, meaning the local clock did not visibly advance since the last update, the logical counter is incremented by one. If the maximum is strictly greater than the previous physical time component, meaning real time has moved forward, the logical counter resets to zero. This is exactly analogous to a Lamport clock's increment-on-every-event rule, except the increment is only needed when physical time itself fails to provide fresh resolution. On receiving a message carrying a remote timestamp (pt_remote, l_remote), the node again reads its local physical clock pt_now, and computes the new physical time component as the maximum of three quantities: pt_now, the node's own previous physical time component, and pt_remote. Whichever of these three is the largest becomes the new physical component. Then the logical counter is set based on which of those three quantities achieved that maximum: if only the local physical clock reading achieved it, the logical counter resets to zero; if the new physical component ties with the previous local physical component and/or the remote physical component, the logical counter becomes one more than the maximum of the relevant previous logical counters, incrementing to break the tie. In every case the counter never decreases, guaranteeing that the merged timestamp is greater than both the sender's timestamp and the receiver's own prior timestamp, which is exactly what preserves the causal ordering guarantee across message exchanges.

How HLC Preserves Causality Without Losing Physical Meaning

The correctness argument for HLC mirrors the classic proof for Lamport clocks, with an added physical-time bound. Because every update rule takes the maximum of all relevant prior timestamps and, when necessary, strictly increases the logical counter, the resulting timestamp of any event is always guaranteed to be greater than the timestamps of every event that causally precedes it. This is the same clock condition that pure logical clocks satisfy: if event A happened before event B, whether through program order on a single node or through a chain of message sends and receives across nodes, then HLC(A) is guaranteed to be less than HLC(B) under the lexicographic comparison described earlier. What HLC adds on top of that guarantee is a provable bound relating the physical component to true wall-clock time. Under reasonable assumptions, such as bounded clock skew between nodes and bounded message transmission delay, the physical time component of any HLC timestamp stays within a small, bounded distance of the actual physical time at which the event occurred. This bound is what makes HLC timestamps usable for real-world purposes: comparing an HLC timestamp to a wall-clock deadline, computing an approximate duration between two events on different nodes, or displaying a human-readable log entry all remain meaningful, something that would be impossible with pure logical counters. It is worth being precise about what HLC does not give you. It does not give exact real-time ordering the way a perfectly synchronized global clock would, and it does not replace stronger causality tracking structures when a system needs to know the complete causal history of every event, not just a total order consistent with causality. HLC provides a single scalar-like value per event that is easy to store, compare, and index, at the cost of only approximating true simultaneity and only capturing causal precedence rather than full causal dependency information.

Real-World Use: CockroachDB, MongoDB, and Beyond

Hybrid Logical Clocks were popularized in production distributed databases because they solve a very concrete engineering problem: these systems need timestamps that can be used both for correctness, ordering transactions consistently across a cluster, and for operational purposes, like expiring old data, garbage collecting multi-version storage, or reporting time-travel query bounds to humans in a way that resembles ordinary calendar time. CockroachDB uses HLC as the foundation of its transaction ordering and its multi-version concurrency control. Every node maintains an HLC, and every read and write operation is timestamped using it. Because the physical component tracks real time closely, CockroachDB can bound how far a node's clock is allowed to drift from the rest of the cluster and use that bound to make guarantees about transaction consistency, while the logical counter ensures that even operations occurring within the same physical tick, or arriving out of physical-time order due to network delay, are still ordered in a way consistent with causality. MongoDB adopted a related mechanism, generally referred to internally as cluster time, that follows the same hybrid principle for ordering operations across replica sets and shards in causally consistent sessions. When a client performs a causally consistent read after a write, the driver propagates a hybrid timestamp so that subsequent operations on any node in the cluster are guaranteed to observe a view of the data at least as recent as what the client already saw, again relying on the same maximum-and-increment update rule to stitch physical time and logical ordering together. Beyond these two systems, the same idea appears wherever engineers need timestamps that double as both an ordering mechanism and an operationally meaningful clock reading, including various distributed logging and event-sourcing frameworks, some consensus and replication implementations, and academic follow-up work that extends the basic HLC idea with tighter bounds or additional metadata.

Frequently asked questions

Is a Hybrid Logical Clock the same thing as a vector clock?

No. A Hybrid Logical Clock produces a single, compact, scalar-like pair per node that is designed to stay close to physical wall-clock time while still respecting causal order between any two events. A vector clock, by contrast, stores one counter per participating node, growing with the number of nodes in the system, and its purpose is to let you determine the full causal relationship between any two events, including detecting when two events are concurrent, meaning neither causally happened before the other. This site's vector-clocks-lab focuses on that full causal-ordering picture using per-node counter vectors, whereas this lab focuses on how HLC achieves a single, humanly meaningful timestamp that still respects causal precedence. HLC does not tell you whether two events are concurrent, only that a happened-before relationship, if one exists, is reflected correctly in the comparison.

Why does the logical counter increment instead of just using physical time directly?

Physical clocks have limited resolution and can be read identically twice in a row, or a received message can carry a timestamp that is not strictly greater than the local physical clock reading even though it represents an earlier or equal moment causally. In both situations, physical time alone cannot distinguish the order of events. The logical counter exists specifically to break these ties in a way that always increases, guaranteeing that no two causally ordered events ever receive the same or an out-of-order HLC timestamp, even when their physical time components happen to coincide.

Can the physical component of an HLC timestamp ever run far ahead of the actual wall clock?

In a well-behaved system with bounded clock skew and bounded message delays, no. The update rule only ever advances the physical component to match the largest physical time value it has observed, either the node's own clock or a timestamp carried by an incoming message. Since messages only carry timestamps that were themselves bounded by the sender's physical clock plus a bounded amount of skew, the physical component of any node's HLC stays within a provably bounded distance of true time. Excessive clock skew or a misbehaving clock on one node can widen this bound, which is why real systems like CockroachDB monitor and enforce maximum allowed clock offsets between nodes.

Do I need synchronized clocks, such as NTP, for HLC to work correctly?

HLC guarantees correct causal ordering even with completely unsynchronized clocks, because the update rule always takes the maximum of observed timestamps regardless of how far apart the underlying physical clocks are. However, the practical benefit of HLC, namely that its physical component stays meaningfully close to real time, depends on the underlying clocks being reasonably synchronized. Systems that use HLC in production still run protocols like NTP to keep node clocks close together, both to keep the logical counter small and to keep the physical component operationally useful for humans and for time-based logic like expiry windows.

How does receiving a delayed or out-of-order message affect an HLC timestamp?

The update rule is designed specifically to handle this gracefully. When a node receives a message, it computes its new HLC as the maximum of its own previous HLC, the timestamp carried in the message, and its current physical clock reading, then applies the appropriate logical counter increment based on which of those values determined the maximum. This means that even a message that arrives late, or whose timestamp is smaller than the receiving node's current physical time, is still correctly incorporated: the receiving node's new timestamp is guaranteed to be greater than both its own prior timestamp and the message's timestamp, preserving causal order regardless of network delay or delivery order.

Try it live

Everything above runs in your browser — open Hybrid Logical Clocks Lab and change the parameters while it is running. Nothing is installed, nothing is uploaded, the whole model lives in one tab.

▶ Open Hybrid Logical Clocks Lab simulation