Inference

Part 10 of the AI Engineer Series. Speculative decoding (Medusa, EAGLE-3) and quantization (AWQ, GPTQ, INT4/INT8) attack different inference bottlenecks. They stack with each other and with paged KV management. The numbers, the failure modes, and when to bother.

Part 9 of the AI Engineer Series. The KV cache is the single largest variable in inference economics most application teams never look at. Memory math with real numbers, PagedAttention and vLLM, prefix sharing, and why long contexts get expensive faster than the pricing page suggests.