Cost Attribution Per Feature: Why "How Much Does Our AI Cost" Is the Wrong Question

The wrong question and the right one

Every CFO eventually asks the same thing: how much does our AI cost? Every AI engineer eventually realizes that is the wrong question. The right question is more specific. How much does the summarization feature cost per active user per month? How much does the support agent cost per resolved ticket? What is the marginal cost of a power user versus a casual one?

Without that level of resolution, you cannot make rational tradeoffs. You cannot know which feature to invest in, which to optimize, which to deprecate. This is Part 8 of the AI Engineer Series, and it covers the tagging discipline that makes those answers possible.

The tagging discipline

Every LLM call your harness makes must carry tags. At minimum:

• feature: the user-facing capability ("summarize", "chat", "search", "extract")
• workflow_step: the specific step in a multi-call workflow ("retrieve", "rerank", "synthesize")
• user_id (hashed): for per-user economics
• tier: free, pro, enterprise. Cost behavior differs wildly by tier.
• environment: prod, staging, dev. Confusing these is how dev traffic ends up on the prod bill.
• experiment: A/B test arm if applicable.

These tags propagate through your trace spans (from Part 5) and end up in your cost data warehouse. The discipline is enforcement: make tagging mandatory at the harness layer so no LLM call escapes without them.

from dataclasses import dataclass
from typing import Literal

@dataclass(frozen=True)
class CallContext:
    feature: str
    workflow_step: str
    user_id: str
    tier: Literal["free", "pro", "enterprise"]
    environment: Literal["prod", "staging", "dev"]
    experiment: str | None = None

def call_llm(prompt: str, ctx: CallContext, model: str):
    response = client.messages.create(
        model=model,
        max_tokens=1024,
        messages=[{"role": "user", "content": prompt}],
    )
    
    # Compute cost from usage and pricing table
    cost = compute_cost(model, response.usage)
    
    # Emit cost event with full attribution
    emit_cost_event(
        feature=ctx.feature,
        workflow_step=ctx.workflow_step,
        user_id=ctx.user_id,
        tier=ctx.tier,
        environment=ctx.environment,
        experiment=ctx.experiment,
        model=model,
        input_tokens=response.usage.input_tokens,
        cached_input_tokens=getattr(response.usage, "cache_read_input_tokens", 0),
        output_tokens=response.usage.output_tokens,
        cost_usd=cost,
    )
    return response

If you cannot add this signature to your call site without effort, you have a harness problem. Fix the harness first.

Token accounting nuances

"Cost" is not one number. Track these separately because they have different price points:

• Uncached input tokens: full input price
• Cached input tokens: typically 10-50% of input price (provider-dependent)
• Cache write tokens: 1.25x or 2x input price on Anthropic, free-ish on OpenAI
• Output tokens: typically 3-5x input price
• Tool call tokens: counted as output
• Thinking tokens: for reasoning models, the hidden reasoning is large and fully billed as output

If you bucket everything as "input + output," your unit economics lie to you. The reasoning-model trap especially. A single thinking-enabled call can produce 5-10x more billable output tokens than the visible response, with no warning if you are not looking.

The unit economics questions you must answer

For every feature, you should be able to look up:

• Cost per successful invocation
• Cost per active user per month
• Cost as a percentage of revenue (for paid tiers)
• Marginal cost of the 100th invocation in a session versus the first
• The top 10 most expensive sessions yesterday, with full traces available

That last one is the most underrated. Looking at the most expensive 10 sessions every day for a week will surface 90% of the cost bugs in your system. Always.

Hidden cost drivers

These are the ones that bite. Watch for them.

Retry storms. A flaky prompt that fails validation 30% of the time costs you 30% extra without changing the user experience. Track schema-failure rate per prompt. A rising failure rate is a cost regression as much as a quality one.

Agent loops. A 10-step agent costs 10x a single call before you count the output. An agent with a runaway loop costs 50x. Budget caps per agent run, covered in Part 12, are mandatory. Without them, one bad query can ruin your daily cost average.

Long-context regressions. A prompt that creeps from 2K to 8K tokens because someone added a few examples quadruples input cost silently. Token-usage drift alerts from Part 5 catch this.

Premium fallbacks gone wrong. Your cheap model fails 10% of the time and falls back to a model 20x more expensive. Effective cost is 3x the cheap model alone. If your fallback rate climbs, you are paying expensive-model prices for cheap-model traffic.

Output verbosity creep. The same prompt produces longer answers after a prompt edit. Cost rises 30-50% with no visible improvement. The defense is a token-budget assertion in your evals: longer outputs without quality lift is a regression.

Budget alerts and circuit breakers

Set budget thresholds at three layers:

• Per-user per-day. Caps abuse, runaway loops, accidental scripts.
• Per-feature per-hour. Catches regressions before they ruin a month.
• Org-wide per-month. The finance-team nightmare prevention.

Pair these with circuit breakers. When a budget trips, fail open: degrade to a cheaper model, return a cached response, queue for later. Failing closed in an LLM system means a partial outage, which is worse than a partial cost overrun.

class BudgetGate:
    def __init__(self, daily_limit_usd: float):
        self.daily_limit = daily_limit_usd
        self.spent_today = {}  # user_id -> usd
    
    def check(self, user_id: str) -> str:
        spent = self.spent_today.get(user_id, 0)
        if spent >= self.daily_limit:
            return "block"
        if spent >= 0.8 * self.daily_limit:
            return "degrade"  # use cheaper model
        return "allow"
    
    def record(self, user_id: str, cost: float):
        self.spent_today[user_id] = self.spent_today.get(user_id, 0) + cost

The dashboard you actually need

One page. Five charts. Refreshes daily.

• Total spend, 30-day trend, with last-week and last-month annotations
• Cost per feature, stacked area over 30 days
• Cost per active user, p50/p95, broken down by tier
• Cache hit rate per prompt template (from Part 7)
• Top 10 most expensive sessions yesterday, linked to their traces

If you have these five, you can answer almost any cost question without an engineer pulling data manually. Build the dashboard once, look at it weekly, and your bill stops surprising you.

What to build this week

Three things, in order:

• Add the CallContext pattern to your harness. Make every call carry feature, user_id, and tier.
• Pipe cost events to a small DuckDB or BigQuery table. Build the five-chart dashboard above.
• Spend 30 minutes looking at your top 10 most expensive sessions from the past week. Pick one cost bug to fix this week.

The third step is where the savings happen. Every team I have seen do this finds something embarrassing in the top 10. A debug log accidentally calling the LLM on every page load. A scheduled job that retries forever. A user who scripted a search loop. The first audit pays for everything else.

What is next

Part 9 starts the inference internals batch. We will look at the KV cache, the data structure that decides how far your GPU budget goes, why long contexts get expensive, and what PagedAttention actually does under the hood.

Previous in the series

• Part 1: Harness Engineering
• Part 2: Context Engineering & Retrieval Quality
• Part 3: Structured Outputs and Fallback Chains
• Part 4: Evals
• Part 5: LLM Observability
• Part 6: Latency and Throughput Engineering
• Part 7: Prompt Caching vs Semantic Caching

References

1. Anthropic Usage & Cost API. The endpoint for pulling cost data with attribution.
2. OpenAI Usage API. Same purpose, OpenAI-side.
3. Anthropic pricing reference. Current input, output, cache read/write, and thinking-token rates.
4. OpenAI pricing reference. Same for OpenAI models.
5. Cost-effective LLMOps by Eugene Yan. Broader patterns for cost management in production.
6. The economic case for generative AI by Andreessen Horowitz. Industry-wide unit economics framing.