MobileRider Answers

How do you allocate LLM costs by team

You allocate LLM costs by team by routing every model call through an LLM gateway that issues each team its own virtual key, then reporting metered spend per key. Provider dashboards cannot do this on their own, because each provider sees one account and one invoice, not the internal teams behind them.

Why provider dashboards cannot do it

OpenAI, Anthropic, Azure OpenAI, and Bedrock all report usage at the account or project level. That answers one question: what did the organization spend with this provider. It cannot answer the question finance actually asks: which team, product, or customer drove that spend.

The problem compounds when you use more than one provider, which most enterprises now do. Each dashboard has its own format, its own granularity, and no shared concept of your org chart. Stitching four provider exports together still gets you totals by provider, not totals by team.

Why tagging fails at the app layer

The obvious workaround is to have engineers tag each call with metadata identifying the team or feature. In practice this decays quickly. It depends on every engineer, on every team, passing consistent tags through every SDK and every provider, forever. New services launch without tags. Naming conventions drift. Batch jobs and internal tools get skipped. And nothing enforces any of it, because the provider will happily serve an untagged call.

Cloud FinOps learned this lesson years ago: attribution that depends on voluntary discipline in application code produces incomplete data. Attribution has to live in the infrastructure layer, where it happens automatically for every request.

The per team key model

The reliable pattern is structural. Deploy an LLM gateway, such as LiteLLM, between your applications and your providers. Then:

1. Retire shared provider keys. Raw provider credentials live only in the gateway.
2. Issue virtual keys per team. Each team, application, or environment gets its own key from the gateway. Granularity is your choice: one key per team is the floor, keys per feature or per customer are common as programs mature.
3. Meter every call. The gateway records model, tokens, computed cost, and the key that made the request. Attribution is a property of the request itself, not an annotation someone remembered to add.
4. Enforce with the same mechanism. Keys carry budgets, alerts, rate limits, and model permissions, so the allocation layer doubles as the control layer.

What good allocation output looks like

The baseline report is spend by team, by model, by day, across all providers in a single view. From there, useful additions include drill down from team to feature to customer, budget consumption per key with alerts before limits are hit, and exports into the FinOps or BI tooling your finance team already uses. When someone asks why a number moved, the answer should trace to specific keys, models, and days, not to a proration formula.

The same discipline cloud went through

This is a rerun of cloud cost allocation from roughly a decade ago. Early cloud bills were one undifferentiated number, engineers resisted tagging, and finance allocated by guesswork until tagging standards, cost centers, and FinOps practice made allocation routine. LLM spend is at the beginning of that same curve. The teams that put per team attribution in place now avoid the painful cleanup phase, and they get pricing leverage and forecasting ability while their peers are still splitting one invoice by headcount.

Frequently asked questions

Why can't I allocate LLM costs with provider dashboards?

Dashboards report at the account level. They cannot see your internal teams, and shared keys make per call attribution impossible after the fact.

Is tagging calls in application code enough?

Rarely. Tagging depends on voluntary, consistent effort across every team and SDK, and it decays. Infrastructure level attribution through per team keys does not.

What does a per team virtual key actually do?

It attributes every call to its owner and carries that team's budget, limits, and model permissions, combining allocation and enforcement in one mechanism.

What should an allocation report show?

Spend by team, by model, by day across all providers, with drill down to feature and customer and exports to your finance tooling.