Sign in Subscribe

Topic

Performance

A collection of 2 issues

Prompt Caching vs Semantic Caching: Two Different Mechanisms Confusingly Named

Part 7 of the AI Engineer Series. Prompt caching and semantic caching sound similar but solve completely different problems. Together they are the largest cost lever in your LLM stack, and confusing them is how teams overspend by 5x.

Latency and Throughput Engineering: The Numbers That Decide if Your AI Feels Alive

Part 6 of the AI Engineer Series. "It is slow" is not a problem statement. LLM latency has structure: TTFT, ITL, and end-to-end. The numbers, the percentiles, and the streaming UX patterns that decide whether your AI feels alive.