LLM token tracking

How to Track LLM Token Usage Per User, Per Feature, and Per Agent

by
LLM token tracking

By Tom Morgan

Picture the invoice. It says you spent a few thousand dollars on model calls last month, up sharply from the month before. Finance asks which product, which customers, which team. Your answer is a shrug, because the only thing the provider told you is the total.

That gap, between “how much” and “whose”, is where AI margins quietly disappear. This guide closes it. It starts with four public cases (one of them shaky, and I’ll say why), compares what the big providers’ usage APIs will and won’t tell you, and then lays out a design you can build in about a week.

Everything here was checked against primary documentation on 2 November 2026. Where I’m giving my own recommendation rather than a documented fact, I say so. Where a number is hypothetical, it’s labelled.

In short: stamp every model call with a user, a feature, and an agent. Record every token type separately. Charge a whole agent tree to the one action that triggered it. Reconcile against the invoice daily. And add a hard cap, because an alert is not a brake.

Four cases that explain why this matters

Case 1: the $200 plan and the user nobody could see (July 2025)

On 28 July 2025, Anthropic told Claude subscribers it would add weekly rate limits from 28 August. The reason it gave was that some people ran Claude Code continuously in the background, and that one user consumed “tens of thousands in model usage” on a $200 plan. The company estimated the limits would affect fewer than 5% of subscribers.

Look at the shape of it. A flat price met a usage distribution with a very long tail. The average looked healthy. A handful of accounts did not. The other half of the story is in the customer reaction: one user complained that without a usage dashboard, staying within limits became guesswork.

What it teaches: per-user attribution protects you in two directions. It shows you the tail before it hits your margin, and it lets you show customers their own consumption, which softens any limit you introduce later.

Case 2: one click, fifteen times the tokens

Anthropic’s engineering team published an unusually frank account of how they built the multi-agent Research feature. The sentence cost owners should read twice: in their data, agents typically use about 4× more tokens than chat, and multi-agent systems about 15× more. The same post reports that a Claude Opus 4 lead with Claude Sonnet 4 subagents beat single-agent Opus 4 by 90.2% on their internal research eval, and says plainly that this only pays off on high-value tasks.

Relative tokens per user action (Anthropic's reported multiples)

Chat          █                        1×
Single agent  ████                     ≈4×
Multi-agent   ███████████████          ≈15×

A caveat before you quote this: those multiples describe Anthropic’s own Research system. Yours will differ. The point isn’t the exact ratio. It’s that one button press can fan out into a tree of calls, and if you only log per request, the tree is invisible.

Case 3: even the vendor’s own “average” moved (April 2026)

Anthropic’s Claude Code cost page currently says the average is around $13 per developer per active day, $150 to $250 per month, with 90% of users under $30 per active day. Earlier in April 2026, the same page said $6 per day, with 90% below $12, according to archived copies reported by the press.

I don’t know why the figure changed, and I won’t guess. The useful reading is simpler. Averages drift as tools and habits change, and the page itself notes that costs vary widely with model choice, codebase size, and how many instances people run. Anyone budgeting from a published average was budgeting from a moving number. Measuring your own usage is the only stable baseline.

Case 4: the $47,000 loop (reported November 2025), with a warning label

This one is widely repeated: four LangChain-style agents coordinating over the A2A protocol, two of them stuck in a back-and-forth for roughly eleven days, and a bill of about $47,000 that nobody noticed until the invoice. Several write-ups describe green dashboards and no errors the whole time.

Treat it as a cautionary tale, not as hard evidence. I traced the story back through retellings and couldn’t verify the original report, and the retellings disagree with each other. Some say eleven days; others describe a four-week escalation. One version’s weekly figures ($127, $891, $6,240, $18,400) add up to about $25,700, and its author notes that the $47,000 total includes other infrastructure costs. So the dollar figure is soft.

The pattern, though, is believable and well documented in principle: agents that talk to each other with no completion rule can iterate indefinitely, and everything looks healthy because nothing technically fails. The practical lesson is the one I’ll come back to in the guardrails section. Monitoring that tells you about a loop on day eleven is not the same as a limit that stops it on day one.

What your provider can and can’t tell you

Both major providers now offer admin-level usage APIs. They’re good. They stop short of “which of my customers”.

ProviderGroup or filter byWatch out for
Anthropic Usage & Cost Admin APIAPI key, workspace, model, service tier, context window. Cost report groups by workspace or description.Requires an Admin API key (organisation accounts only). Default-workspace usage shows a null workspace_id. Workbench usage has a null api_key_id. Priority Tier costs aren’t in the cost endpoint.
OpenAI Usage API (completions)Project, user, API key, model, batch, service tierThe user_id field belongs to OpenAI’s organisation model. Test in a scratch project whether it maps to your end customers. I’d assume it doesn’t.
Anthropic Claude Code Analytics APIDaily, per-user records for Claude Code, with token and estimated cost data, keyed to a user or API keyDaily aggregates, not real time. For real-time, per-user streams Anthropic points to OpenTelemetry export. Claude Code only, not your own app.

Two practical points follow. First, workspace or project splits work for ten internal teams and break at ten thousand customers, because you can’t mint a key per user. Second, the provider’s numbers are the authority on what you owe, which makes them the right thing to reconcile against, but they can’t be the thing that answers “whose”.

One more detail worth knowing: usage and cost data typically arrives within about five minutes of a request completing, so it’s fine for daily reconciliation and too slow for a safety brake.

So you need your own ledger.

The design

Here’s the architecture in one picture. Everything on the left of the ledger is yours; everything on the right is the provider’s truth you check against.

the architecture

1. One row per model call

Every call writes one row. Nothing else is required to start.

text

ts, request_id, root_request_id, parent_request_id,
user_id, tenant_id,            -- who
feature,                       -- "chat", "doc-summary", "research"
agent,                         -- "lead", "search-subagent", "critic"
model, provider,
input_tokens, output_tokens,
cache_read_tokens, cache_write_tokens, reasoning_tokens,
cost_usd_estimated, latency_ms, status

The root_request_id is the column most teams forget, and it’s the one that makes agent trees accountable. Every call in the tree carries the ID of the user action that started it. Sum by root and you get the true cost of one click, which is what Case 2 says you otherwise can’t see.

2. Use standard names for the numbers

OpenTelemetry’s GenAI conventions already define the token fields. The core attributes include gen_ai.request.model, gen_ai.usage.input_tokens and gen_ai.usage.output_tokens, with finer ones like gen_ai.usage.cache_read.input_tokens and gen_ai.usage.reasoning.output_tokens for cached and reasoning tokens. The older prompt_tokens and completion_tokens names were renamed, so check which your instrumentation emits.

Two honest caveats. A mid-2026 guide notes that the chat and embeddings conventions are mature while agent and tool-orchestration ones are still marked Development. And one secondary source reports the GenAI conventions moved to their own repository in June 2026. Pin a version and read the official spec before you commit.

3. Add the three tags the standard doesn’t give you

User, feature, and agent are your business context, so they’re custom attributes. I’d namespace them app.user_id, app.feature and app.agent, set them once at the edge, and let context carry them down. That way the sixth subagent never has to remember who it’s working for.

python

from opentelemetry import baggage, context, trace

tracer = trace.get_tracer("llm-ledger")

def bind_request(user_id: str, feature: str, root_id: str):
    """Call once at the API edge. Returns a token to detach later."""
    ctx = baggage.set_baggage("app.user_id", user_id)
    ctx = baggage.set_baggage("app.feature", feature, context=ctx)
    ctx = baggage.set_baggage("app.root_id", root_id, context=ctx)
    return context.attach(ctx)

def call_model(client, *, model, messages, agent):
    with tracer.start_as_current_span(f"chat {model}") as span:
        span.set_attribute("app.user_id", baggage.get_baggage("app.user_id"))
        span.set_attribute("app.feature", baggage.get_baggage("app.feature"))
        span.set_attribute("app.root_id", baggage.get_baggage("app.root_id"))
        span.set_attribute("app.agent", agent)

        resp = client.messages.create(model=model, max_tokens=1000,
                                      messages=messages)
        u = resp.usage
        span.set_attribute("gen_ai.request.model", model)
        span.set_attribute("gen_ai.usage.input_tokens", u.input_tokens)
        span.set_attribute("gen_ai.usage.output_tokens", u.output_tokens)
        return resp

This sketch is written against the documented OpenTelemetry and Anthropic SDK shapes but I haven’t run it in your stack, so test it before trusting it. Baggage travels automatically inside one process and across async tasks that inherit context. Across services, you’ll need the W3C propagators configured. And make the wrapper the only way to reach the SDK: if developers can call the client directly, your tags will have holes within a month. A lint rule that bans direct calls is cheap insurance.

4. Or let a gateway do the tagging

If you’d rather not touch every call site, route traffic through a proxy. LiteLLM is the documented open-source example. Its docs describe spend tracking per key, user and team, with tags passed in request metadata, and its database keeps pre-aggregated daily rollups per user, team, org, tag, end user and agent. Its Prometheus endpoint exposes spend and token metrics labelled by end user, team, model and more. In the docs I read, tag-based spend was flagged as an Enterprise feature, so check the licence before you plan around it.

Observability platforms attack the same problem from the SDK side. Langfuse’s documentation says that passing a user_id enables per-user cost and token views, that identifiers can be propagated to every child observation, and that per-user metrics can be queried through a Metrics API.

ApproachStrengthWeakness
In-code wrapper and your own ledgerFull control, agent-level detail, parent/child linksYou maintain it, and every call path must go through it
Gateway such as LiteLLMOne choke point, budgets and alerts includedExtra hop and dependency. Agent identity is only as good as the tags you send
Observability platform such as LangfuseDashboards and per-user views out of the boxAnother system to run or pay for, and tags still come from your code
Provider admin API aloneZero code, matches the invoiceNo per-customer view unless you split keys or workspaces

My recommendation for most teams: a thin in-code wrapper that writes to your own table, plus the provider API for reconciliation. Add a gateway when you need enforced budgets across many services.

A worked example (hypothetical numbers)

Real companies rarely publish their internal breakdowns, so here is an illustration. Every figure below is invented for the sake of the arithmetic, and the price is an assumed $3 per million input tokens and $15 per million output tokens. Swap in your own.

FeatureRequests/dayAvg tokens (in / out)Cost per requestDaily cost
Chat20,0006,000 / 800$0.030$600
Document summary3,00030,000 / 1,000$0.105$315
Multi-agent research400150,000 / 12,000 (whole tree)$0.630$252
Total23,400$1,167

Now compare the shares:

FeatureShare of requestsShare of spend
Chat85.5%51.4%
Document summary12.8%27.0%
Multi-agent research1.7%21.6%

Research is under 2% of traffic and over a fifth of the bill. A per-request dashboard would show it as noise. A per-root-action ledger shows it as the line item to price, cap, or gate behind a higher plan. That reframing, from “requests” to “user actions”, is the whole value of the root_request_id column.

Queries you’ll actually run

Three views cover most decisions. These are Postgres-flavoured; adapt to your warehouse.

sql

-- 1. Where is the money going?
SELECT feature, agent, model,
       SUM(input_tokens) AS in_tok, SUM(output_tokens) AS out_tok,
       SUM(cost_usd_estimated) AS cost
FROM llm_ledger
WHERE ts >= now() - interval '7 days'
GROUP BY 1,2,3 ORDER BY cost DESC;

-- 2. Who is in the tail?
SELECT user_id, SUM(cost_usd_estimated) AS cost
FROM llm_ledger
WHERE ts >= date_trunc('day', now())
GROUP BY 1 ORDER BY cost DESC LIMIT 20;

-- 3. How much of the bill can't you attribute?
SELECT SUM(cost_usd_estimated) FILTER (
         WHERE user_id IS NULL OR feature IS NULL OR agent IS NULL)
       / NULLIF(SUM(cost_usd_estimated), 0) AS unattributed_share
FROM llm_ledger
WHERE ts >= now() - interval '1 day';

That third query is the one I’d put on the wall. Call it attribution coverage: the share of spend that carries all three tags. If it’s 70%, your per-feature numbers are fiction for 30% of the money. Aim for coverage above 95% and treat every drop as a bug. (That threshold is my rule of thumb, not an industry standard.)

Traps that burn people

Putting user_id on a metric label. In Prometheus-style systems, every distinct label value becomes its own time series, so a per-user label can explode at scale. This is my engineering judgement rather than a documented rule for these products: keep per-user detail in the ledger or warehouse, and keep dashboards on feature, agent, and model, where the label count stays small.

Double-counting cached tokens. Providers disagree on whether the input count already includes cached tokens. Langfuse’s cost guide describes an exclusive-buckets contract precisely because mixing the two can price the same tokens twice. Read your provider’s usage object carefully, store cache reads, cache writes and reasoning tokens as separate columns, and price each at its own rate.

Ignoring failed calls. A call that errors after generating output can still cost money. Store status next to tokens so you can see waste, not only spend.

Trusting your price table. Your per-call cost is an estimate. Anthropic itself describes its Claude Code figures as estimates and treats the Console as the authoritative usage record. Once a day, compare your ledger total to the provider’s cost report. A persistent gap usually means an untagged call path, a stale price, or a token type you aren’t recording. I’d investigate anything beyond a few percent.

Logging prompts by default. Token counts aren’t sensitive. The user ID next to them is personal data, so pseudonymise it in analytics. And keep prompt-content capture off unless you need it; the OpenTelemetry conventions treat message content as opt-in for good reason.

Guardrails: an alert is not a brake

Case 4’s real lesson is about the gap between knowing and stopping. A dashboard that turns red on day eleven has done its job and still lost you the money. A few controls change the outcome, and they all sit in the same wrapper that writes your ledger:

  • Charge the root, cap the root. Give every user action a token or dollar budget, and have subagents draw from it. When it’s spent, the next call fails fast instead of the tree growing.
  • Cap iterations and set a completion rule. Two agents that can ask each other for clarification forever eventually will. A hard maximum on turns per root action turns an infinite loop into a bounded one.
  • Daily ceilings per user and per feature. Cheap to implement with a counter keyed on user and day, checked before each call.
  • Use the provider’s own limits as a backstop. For example, Claude Code on the API supports workspace spend limits and per-user rate-limit guidance. They’re blunt, but they work even if your code has a bug.
  • Alert on change, not only on level. A user crossing several times the median, or a feature whose daily cost jumps sharply, is a signal long before any absolute threshold.

python

def check_budget(root_id: str, user_id: str, est_tokens: int):
    spent = redis.incrby(f"tok:{root_id}", est_tokens)
    if spent > ROOT_TOKEN_LIMIT:
        raise BudgetExceeded("root action over token budget")
    if redis.incrby(f"tok:{user_id}:{today()}", est_tokens) > USER_DAILY_LIMIT:
        raise BudgetExceeded("user over daily budget")

Estimate before the call, correct with the real count after, and fail closed on the budget check. It’s a sketch, not a finished system, but even this crude version would have capped the worst-case in a runaway loop.

A one-week build plan

  1. Day 1. Agree the three tags and the ledger schema. Argue about feature names now.
  2. Day 2. Route every model call through one wrapper. Block direct SDK calls with a lint rule.
  3. Day 3. Set user, feature and root ID at the API edge using request-scoped context.
  4. Day 4. Propagate to subagents and tools. Store parent and root IDs so a tree rolls up to one action.
  5. Day 5. Build three views: cost per feature per day, top 20 users today, cost per agent role.
  6. Day 6. Reconcile against the provider’s cost report. Chase the gap until coverage and totals agree.
  7. Day 7. Add the budget guard, the iteration cap, and the change-based alerts.

Frequently asked questions

Can’t I just use the provider’s dashboard? For the total and for splits by key, workspace, or model, yes. For per-customer or per-feature answers, you need your own tags, because the provider only knows what you sent in the request’s credentials, not who your customer was.

Is a gateway better than instrumenting my code? A gateway gives you one choke point and built-in budgets, at the cost of another dependency. Instrumenting your code gives finer agent detail. Many teams end up with both. Whichever you choose, you still need to send the user, feature, and agent tags.

How do I track cost per agent in a multi-agent system? Tag each call with an agent role and a shared root action ID. Per-agent cost is a group-by on the role. Per-action cost, which is usually what pricing decisions need, is a group-by on the root.

Should I store prompts and responses too? Only if you need them for debugging or evaluation, and then with retention limits and access controls. For cost tracking, token counts and metadata are enough.

How accurate will my estimates be? Close, if you record every token type and keep your price table current, but never perfect. That’s why the daily reconciliation against the provider’s cost report is part of the design, not an extra.

The short version

The cases point the same way. Flat prices meet long-tailed usage (Case 1). Agent systems multiply tokens in ways a request log can’t show (Case 2). Published averages shift under you (Case 3). And a loop with no ceiling can run for days while every dashboard stays green (Case 4, with the caveats above).

Provider dashboards answer “how much”. Only your own ledger answers “whose, which feature, which agent”. So tag every call, record every token type, charge agent trees to the action that started them, reconcile daily, and put a hard cap in the same code path. That is the whole discipline, and none of it needs exotic tooling.

Sources

Last verified against the linked documentation on 2 November 2026. Provider APIs, prices, and conventions change; if a link or a field name has moved, trust the current official docs over this page.