Vivek Wisdom

Sign in Subscribe

Latest — 13 Sep 2026

Hypothetical Answers, Real Retrieval — How HyDE Fixes the Query-Document Mismatch in RAG

Standard RAG embeds a short question and matches it against long, declarative documents, and that mismatch quietly breaks retrieval more often than teams realize. HyDE fixes it by retrieving with a hypothetical answer instead of the raw query.

More issues

Agent-to-Agent Authorization: What A2A Actually Guarantees (and What It Quietly Hands Back to You)

AAIF just gave MCP and A2A the same governance home. That's a coordination story, not a security fix. Here's what A2A's spec actually guarantees about agent-to-agent authorization, and the three holes you still have to close yourself.

Fine-tune vs In-Context Learning: The Decision Framework That Actually Works

Part 13, the final post in the AI Engineer Series. The decision between fine-tuning and in-context learning is almost always less obvious than it seems. LoRA, QLoRA, DPO, the eval-driven framework, and the signs that mean "fine-tune now" versus "your prompt has more headroom."

Agent Guardrails and Loop Budgets: How to Keep Agents From Ruining Your Week

Part 12 of the AI Engineer Series. An agent without budgets is a bug waiting to be found in production. The five budgets every run needs, tool-use safety principles, prompt injection defenses, and the OWASP LLM Top 10 items that actually matter for agents.

Model Routing and Graceful Fallback: Picking the Right Model for Each Request

Part 11 of the AI Engineer Series. Not every request needs your best model. Cascading, classifier-based dispatch, learned routers (RouteLLM), and the provider-failover pattern that keeps you alive when your primary goes down. The order to build them in, and the metrics that tell you it's working.

Speculative Decoding and Quantization: The Two Levers That Compound

Part 10 of the AI Engineer Series. Speculative decoding (Medusa, EAGLE-3) and quantization (AWQ, GPTQ, INT4/INT8) attack different inference bottlenecks. They stack with each other and with paged KV management. The numbers, the failure modes, and when to bother.

The KV Cache: The Data Structure That Decides Your Inference Economics

Part 9 of the AI Engineer Series. The KV cache is the single largest variable in inference economics most application teams never look at. Memory math with real numbers, PagedAttention and vLLM, prefix sharing, and why long contexts get expensive faster than the pricing page suggests.

Cost Attribution Per Feature: Why "How Much Does Our AI Cost" Is the Wrong Question

Part 8 of the AI Engineer Series. "How much does our AI cost?" is the wrong question. "How much does Feature X cost per user per month?" is what you actually need to know. Tagging discipline, unit economics, and the hidden cost drivers behind your monthly bill.

Prompt Caching vs Semantic Caching: Two Different Mechanisms Confusingly Named

Part 7 of the AI Engineer Series. Prompt caching and semantic caching sound similar but solve completely different problems. Together they are the largest cost lever in your LLM stack, and confusing them is how teams overspend by 5x.

Latency and Throughput Engineering: The Numbers That Decide if Your AI Feels Alive

Part 6 of the AI Engineer Series. "It is slow" is not a problem statement. LLM latency has structure: TTFT, ITL, and end-to-end. The numbers, the percentiles, and the streaming UX patterns that decide whether your AI feels alive.