AxisOps Harbour
Labs

Topics

inference

The model fits; the memory bus decides

Quantised models running on a workstation, a Pi or a phone. The interesting constraint is memory bandwidth rather than FLOPS, which changes which hardware is worth your money, and a capable model on a machine you own means no prompt ever leaves the building.

The expensive part stopped being the munition

A first-person-view quadcopter with a warhead taped to it costs a few hundred pounds and destroys a vehicle worth several million. The airframe is a toy, the radio is a toy, and the only component that was ever hard is now a model that fits on a board you can buy. That is a cost inversion, and cost inversions do not reverse.

The protein problem became a data problem

A neural network won CASP13 outright, and a sequencer that runs off a laptop's USB port costs less than a server. Biology is becoming a field where the expensive instrument is a model and the cheap instrument fits in a coat pocket, which changes who is able to do the work and who ends up holding the data.