Most of us met large language models through an HTTP API. That abstraction holds until the first latency complaint. A support bot that answered in 900 ms starts taking four seconds after someone adds a long policy document to the system prompt. A summarisation job that was cheap becomes expensive when we ask for longer outputs. A self-hosted model that handled 40 concurrent users falls over at 60 even though GPU utilisation looks low. All of these come from a few mechanics inside a decoder-only...