A box of unboxed kit, 2015. The physical layer kept right on existing throughout a decade of people explaining that it had been abstracted away.
Each wave was a real improvement that created the next wave's problem.
Wave one: virtualise the machine¶
VMware, Xen, KVM. One physical server becomes many logical ones. The win was utilisation; physical servers were running at single-digit percentages because every application wanted one of its own.
What it didn't change was the unit. A VM is still a machine. It has a kernel, an init system, a package manager, a hostname and somebody whose job it is to patch it. You now had more machines than you started with, which is why configuration management (CFEngine, Puppet, Chef, later Ansible) rose in this period: the machine count had outgrown hand-tending.
The new problem: VM sprawl. Provisioning got easy and de-provisioning never did. Rather like a garage: getting things in is effortless, and nobody ever takes anything out.
Wave two: abstract the machine away¶
OpenStack's ambition was an open API over pooled compute, storage and network, so that a machine became an API call.
The API argument has won completely. OpenStack itself largely hasn't, and the reason is instructive: the operational burden of running it crept up on the burden it was supposed to remove. A control plane with that many services, each with its own database and message queue, is a distributed system that needs its own team.
The new problem: the unit was still a machine, so you still had machines to patch. You just patched them via an API.
Wave three: schedule against a pool¶
Containers plus an orchestrator. Now the unit is a process with its dependencies, and the machine really does recede: you declare what should run and a scheduler places it.
This one is sticking, and it's a real advance. What the declarative model changed: the system has a desired state written down, and a controller continuously reconciling towards it. That's the durable idea, far more than containers ever were.
The new problem, and it's a subtle one: the abstraction is excellent right up until it leaks, and when it leaks you need to understand every layer below it at once. Kernel, container runtime, the network plumbing, the scheduler's own control loops. The operational floor is higher than before, not lower. Teams adopting it in search of simplicity are generally not finding any.
What reconciliation makes possible¶
Take the declarative loop seriously and some long-standing expensive things get cheap.
Recovery stops being a procedure. Today a failed host means a runbook, a person and an hour; under a reconciling controller it means the controller notices the declared state is unmet and places the work elsewhere. The 3am call becomes a graph you look at on Monday. That is the single largest quality-of-life change available to anyone carrying a pager, and it is available now rather than in five years.
It also puts real infrastructure within reach of organisations that cannot staff a team for it. A small council, a charity, a two-person engineering shop: the thing that used to stop them running their own systems was the number of humans needed to keep twelve machines patched and consistent, not the hardware cost. Declared state plus a reconciler is a real substitute for headcount, and that is the first time it has been true. Our bet is that this is what lets self-hosting come back after fifteen years of losing to convenience.
For engineering the class of bug that dies is configuration drift. The machine that works because somebody fixed it by hand in 2013 and nobody knows what they did has been the standard shape of operational risk for a generation. A system whose state is written down and continuously enforced does not accumulate those.
Who owns the pool¶
The reconciliation model is open and the schedulers are open source, so the chokepoint sits elsewhere: the control plane offered as a service.
Watch what is being built: the orchestrator offered as a managed endpoint, with the cloud's own load balancer, its own identity system, its own storage classes and its own network plugin wired in beneath. Each of those integrations is reasonable on its own and the sum is an orchestrator that only reconciles against one company's infrastructure. The declarative file you wrote looks portable and is not, because every resource it names resolves to a proprietary service. That is a far better lock-in than VMware ever managed, because it arrives as convenience rather than as a licence.
The surveillance consequence needs stating plainly. A managed control plane knows every workload you run, every image you deploy, every secret you mount and every scaling event, which together describe your business in more detail than your accounts do. Three companies holding that for most of the economy is a concentration of commercial intelligence with no precedent, and it is available to any government that can compel disclosure from them.
Here at AxisOps we build the version you can own: your hardware or someone else's, the same declared state, the controller running where you can read its logs. We are against the shape of this technology that only works if you rent it. The open implementations are keeping up today, and keeping up is not automatic; it depends on people continuing to run them on their own kit.
And the declaration can still lie¶
Legibility.
A 2005 server was knowable. You could log in and read the whole thing: which processes, which config files, which cron jobs. Tedious, certainly, but bounded.
A modern cluster isn't knowable that way by any one person. The compensation is supposed to be that it's declared rather than inspected, and that holds only while the declaration really is the source of truth. The moment anybody makes a change outside it, the system is in a state nobody has written down, and nothing whatever announces the fact.
That's the cost of the trade, and it's why the discipline that matters most in this model has nothing to do with tooling: it is the refusal to make manual changes.