inference-engineering

The 150x model tax

How inference techniques either move an LLM deployment along the latency-throughput frontier or push the whole frontier outward.