Clock synchronisation is unglamorous, everything above it quietly depends on it, and the way it fails is rarely the way anyone expects.
What the protocols give you¶
NTP over a normal network gets you within a few milliseconds of reference, sometimes rather worse. That's fine for log correlation and for certificate validity, and no use at all for ordering events between machines.
PTP (IEEE 1588) gets you to microseconds or better, by timestamping in hardware at the network interface and accounting for path delay. It needs network equipment that joins in (transparent or boundary clocks). Across gear that doesn't, the error grows with topology, load and path asymmetry, and can land anywhere from better-than-NTP on a quiet LAN to no better at all. Whether that still clears your accuracy target is the question to ask before ordering the switches. The protocol without the participating network often buys much less than the invoice suggests.
GPS or atomic references anchor the whole thing to something that doesn't drift, which is what you want when the accuracy has to be absolute rather than relative.
The mistake that causes the bugs¶
Using a timestamp as an ordering.
Two events on two machines, timestamped 5ms apart, clocks synchronised to ±3ms. The timestamps say A then B. The truth might be either order. Better synchronisation never removes this; it only shrinks the window in which it's true.
Worse, clocks don't only drift, they step. An NTP correction can move a clock backwards. Code that assumes monotonically increasing time will then see a negative duration, and the usual symptom isn't a crash. It's a nonsensical metric, or an infinite retry, at three in the morning.
The right habits here are boring and absolute:
- Measure durations with a monotonic clock
(
CLOCK_MONOTONIC,time.monotonic()). It can't step and it can't run backwards. Keep wall-clock time for showing times to humans and for things that really are about calendar time. - Don't use timestamps to order events across machines. Use a logical clock, Lamport or vector, if you need ordering; use a consensus protocol if you need agreement.
- Treat a timestamp as an interval rather than a point, if you must use one. That's what Spanner's TrueTime formalises: it reports its uncertainty out loud and waits it out instead of pretending it isn't there.
Where sub-microsecond really matters¶
- Financial trading, where regulation requires timestamp accuracy and sequence is money.
- Telecoms, where radio synchronisation is a physical requirement.
- Industrial control and audio, where jitter is the thing you perceive.
- Distributed tracing, where the span tree is rebuilt from timestamps and clock skew gives you spans that appear to start before their parent.
That last one is the case most engineers meet, usually without realising the negative-duration span they're staring at is a clock problem rather than a tracing bug.
Hardware timestamping used to mean a specialist card and a specialist budget. It is becoming a line item on ordinary server NICs, and that changes what an ordinary engineering team can build.
Databases that can tell you the truth about order. TrueTime made externally-consistent distributed transactions possible, and it needed a private fleet of GPS receivers and atomic clocks in every datacentre. Once a bounded-uncertainty clock is a property of commodity hardware, that capability stops being something one company can offer as a service and becomes something you can build on a rack you own. A distributed database with real serialisable semantics across sites, without renting it: that is a different set of architectures available to a mid-sized business.
Observability that stops lying. The negative-duration span is a daily annoyance now. With microsecond-bounded clocks across a fleet, a trace becomes a measurement rather than a reconstruction, and the whole category of "is this a real latency spike or clock skew" disappears from on-call. That is a class of bug gone, not reduced.
Networks that schedule. Time-sensitive networking only works if the endpoints agree on the microsecond. Get that cheaply and you can run industrial control, professional audio and factory robotics over ordinary switched Ethernet instead of a proprietary fieldbus. A small manufacturer gets to buy commodity network gear for a job that currently requires a vendor's closed ecosystem, and the engineers maintaining it get to use tools they already know.
The prediction we will make is that bounded uncertainty becomes a thing clocks report by default rather than a Google-scale luxury, and that the interval replaces the point as the normal representation of a timestamp in new protocols. The arithmetic is too useful to stay rare.
Precision does not remove the need to trust whoever supplies the time.
Your time comes from somewhere, and almost every estate's answer is a GNSS constellation operated by a military, relayed through a public NTP pool maintained by volunteers or handed to you by whichever cloud you are renting. Each of those is a dependency you did not negotiate. GNSS is spoofable and jammable, demonstrably so and at low cost. A clock you can move is a protocol you can break: certificate validity, token expiry, replay windows and the ordering guarantees above all rest on it. An adversary who can shift your clock by an hour does not need to break your cryptography.
The concentration risk is sharper than people assume. If the regionally authoritative time for a large slice of an industry is one provider's service, then that provider is in a position to define when things happened, and to be compelled to. Time is a surveillance surface too: a high-resolution, authenticated time source correlated across services makes activity attribution much easier, which is useful for forensics and equally useful for tracking people.
The open side is healthy and underused. NTS gives you authenticated time, and chrony and the public NTP pool are good work. A disciplined local reference, a GPS module and a modest oscillator is a few hundred pounds and an afternoon. It means your estate's notion of time survives a dropped upstream outright, and a jammed sky for exactly as long as the oscillator's holdover: a number you have to specify and test rather than assume. We think every serious estate should hold its own time, for the same reason it should hold its own DNS. A dependency you cannot inspect is one you cannot reason about, and you should not have to ask a hyperscaler what o'clock it is.
Report the interval, not a point¶
The discipline is to make a property explicit and checkable rather than assumed. TrueTime's uncertainty interval reports what is actually known. A bare timestamp claims a precision it hasn't got, like a recipe that says "bake until done".