The model fits; the memory bus decides
Quantised models running on a workstation, a Pi or a phone. The interesting constraint is memory bandwidth rather than FLOPS, which changes which hardware is worth your money, and a capable model on a machine you own means no prompt ever leaves the building.
