LLM Token Economics: Cost per Token, Unit Economics and Gross Margin of AI Features (2026)

Per-token prices have collapsed, yet plenty of AI features still lose money on their heaviest users. Both facts are true because a price list quotes dollars per million tokens, while a business earns margin per user, per task, or per outcome. The distance between those two units is where LLM token economics lives, and it is where most pricing mistakes are made.

A single chat turn is cheap. An agent that loops eight times, re-reads a growing context at every step, and burns thousands of hidden reasoning tokens is a different cost object entirely, often ten to fifty times the cost of the “same” feature described as one call. If you price a seat at a flat monthly fee without modeling that distribution, a minority of users can consume the entire margin of the cohort.

This post gives you the model: how tokens are billed, how to turn a price list into cost per request, session and user, how to compute gross margin with a worked example, and which levers cut cost at what quality risk. Prices quoted are dated and sourced; the worked example is illustrative.

What this covers: the billing anatomy of a request, input versus output versus cached versus reasoning tokens, agent-loop cost multiplication, spreadsheet-style margin formulas, pricing-model choices (seat, usage, outcome), and the optimization levers ranked by quality risk.

Context and Background

Every major provider bills text by the token, with separate rates for input and output, and a discounted rate for input that hits a prompt cache. As of the pricing pages I read on 9 November 2026, Anthropic lists Claude Sonnet 5.5 at $2 per million input tokens and $10 per million output tokens, with cache reads at $0.10 per million, and Haiku 5.5 at $0.10 input and $0.50 output for prompts up to 100K tokens (Anthropic pricing). OpenAI’s pricing page lists, for short context, gpt-6.1-sol at $2.00 input, $0.10 cached input and $10.00 output, and gpt-6-luna at $0.10, $0.01 and $0.50 (OpenAI API pricing). Treat every figure here as a dated snapshot: prices change often, and you should re-read the live page before putting numbers into a contract or a board deck.

Two structural facts hold across providers and across years. First, output tokens cost several times more than input tokens, in the lists above roughly five times. Second, there is now a three-tier input price: fresh input, cache write, and cache read, where a cache read costs a small fraction of fresh input. That shape rewards architectures that keep a long, stable prefix and punishes ones that shuffle the prompt on every call.

The economics literature around this is thin, because the unit that matters to a business is not the token. A finance team asks what one active user costs per month, what one resolved ticket costs, and what gross margin the feature contributes. A token price answers none of those directly. It is an input to a cost model, much as an instance-hour price is an input to a cloud cost model.

If you already run FinOps for cloud, the discipline transfers well. Our posts on cloud cost anomaly detection and the FinOps FOCUS billing specification cover the instrumentation mindset. The difference with LLM spend is that cost is driven by the content of the request, not by provisioned capacity, so the same endpoint can cost a hundred times more on one call than another.

There is also a pricing-regime point worth stating early. Frontier prices have fallen sharply for a given capability level, but products have responded by consuming more tokens per task: longer contexts, tool use, multi-step agents, extended reasoning. Per-token deflation and per-task inflation run at the same time. The net effect on your margin depends on which curve your product sits on, and you only find out by measuring.

The Billing Anatomy of One LLM Request

The direct answer: an LLM request cost is the sum of four meters. Fresh input tokens at the full input rate, cached input tokens at the cache-read rate, cache-write tokens at a premium rate when you create a cache entry, and output tokens, including hidden reasoning tokens, at the output rate.

That sentence is the whole bill. Everything else in this post is about predicting how many tokens fall into each meter for your product, and what each meter’s rate is.

Figure 1: The four meters of a single request. The cached prefix, fresh input and output (including thinking) are priced separately, then summed into cost per request.

The diagram shows why prompt layout is an economic decision. Anything in the stable prefix, such as the system prompt, tool definitions and few-shot examples, can be billed at the cache-read rate after the first call. Anything that changes per request, such as the user message and retrieved passages, is billed at the full input rate. Reasoning tokens ride on the output meter, which is the expensive one.

Input tokens: fresh, cache write, cache read

Fresh input is the baseline. On Anthropic’s published multipliers, a five-minute cache write costs 1.25 times the base input price, a one-hour write costs 2 times, and a cache read costs 0.1 times for standard models, with lower multipliers on some newer models (0.05 times on Opus 5.5 and Sonnet 5.5 according to the same page). The page also states that a five-minute write pays for itself after one cache read and a one-hour write after two (Anthropic pricing). OpenAI lists a separate cached-input rate on its pricing page, for example $0.10 against $2.00 fresh for gpt-6.1-sol, which is a 20-to-1 ratio.

The break-even logic is simple. If a prefix of P tokens is reused N times within the cache lifetime, the cost with caching is the write premium plus N-1 discounted reads, compared with N fresh reads. For a five-minute write at 1.25 times and reads at 0.05 times, caching wins from the second call. We worked through the provider differences, TTLs and break-even tables in LLM API prompt caching economics, so this post treats the cache read rate as a parameter rather than re-deriving it.

Output tokens: the expensive meter

Output is generated one token at a time, and providers charge for it at a multiple of input, five times in the Sonnet 5.5 and gpt-6.1-sol lists above. That asymmetry has a practical consequence: shortening the answer saves more than shortening the question. A feature that returns a 1,200-token essay where a 150-token structured answer would do is paying eight times the output meter for no user value.

Reasoning tokens: billed, often invisible

Reasoning (extended thinking) models generate internal tokens before the visible answer. On Anthropic’s platform, thinking is billed as output tokens, the usage object reports the thinking portion in a thinking_tokens field under output_tokens_details, and thinking tokens count toward the max_tokens limit (extended thinking docs). The same page notes that on newer models, prior turns’ thinking blocks are kept in context and billed as input. OpenAI’s pricing page, by contrast, does not itself describe reasoning-token billing in the part I could read, so confirm it in the model documentation for the specific model you use before you build a forecast.

The key modeling point is that visible output length no longer predicts output cost. A request that shows 500 words to the user can have bill-relevant output several times larger. I come back to this with numbers in the agent section.

Batch, tiers and data residency

Batch endpoints discount both input and output. Anthropic lists a 50 percent batch discount on input and output, and says batch and caching multipliers can be combined; it also lists a 1.1 times multiplier for US-only inference on 4.6 and later models. OpenAI’s page shows a batch-style tier at half the standard rate for the example I read, though the tab labelling on the fetched page was ambiguous, so verify it. The architecture side of batch, including when self-hosting beats provider batch, is covered in offline batch LLM inference.

Long-context surcharges

Some price lists have a step function at a context threshold. Anthropic’s page lists Haiku 5.5 at $0.10 per million input tokens for prompts up to 100K tokens and $0.50 above that, counting cache reads and writes toward the threshold. A step like this means a retrieval feature that quietly grows its context from 90K to 110K tokens does not get 22 percent more expensive; its input rate is multiplied by five on that model. Always check whether your model has a size-tiered price before you design around a long window.

From a Price List to Unit Costs

The core move in LLM token economics is to convert provider prices into the units finance cares about. Start with a request-level formula, then roll it up to session, user and cohort.

The request cost formula

Define, for one request: F = fresh input tokens, C = cache-read tokens, W = cache-write tokens, T = thinking tokens, V = visible output tokens. Define prices per million tokens: p_in, p_read, p_write, p_out. Then:

cost_request = (F * p_in + C * p_read + W * p_write + (T + V) * p_out) / 1,000,000

In a spreadsheet, that is one row per request class with six token columns and four price cells, and the cost column is a SUMPRODUCT. You do not need a model for this; you need a log. If you emit token usage as span attributes, you can build the table from traces, and the OpenTelemetry GenAI semantic conventions define standard attributes for it. The upstream spec is at opentelemetry.io.

A worked request example (illustrative)

Take a support-copilot feature on Sonnet 5.5 list prices ($2 input, $10 output, $0.10 cache read per million, from the pricing page above). Assume, as an illustration, that each request carries 3,000 tokens of stable system prompt and tool definitions, 3,000 tokens of fresh input (ticket text plus retrieved articles), and produces 500 visible output tokens with no extended thinking.

Without caching: 6,000 input tokens at $2 per million is $0.012, and 500 output tokens at $10 per million is $0.005, so $0.017 per request.

With the 3,000-token prefix served from cache: 3,000 cache-read tokens at $0.10 per million is $0.0003, 3,000 fresh tokens is $0.006, and output stays $0.005, so $0.0113 per request. That is a 34 percent reduction, and notably the input line fell from $0.012 to $0.0063 while output, the biggest line, did not move at all. Cache-write premiums for the prefix are ignored here because they amortize over many reads; include them if your traffic is bursty and the cache expires between calls.

The lesson is that once the prefix is cached, output tokens dominate the bill on chat-shaped features. Shortening output is then your next most valuable lever, which we revisit under optimization.

Rolling up: session, user, cohort

A user does not make “a request”; they make a distribution of sessions. The roll-up is:

cost_user_month = sum over session types of (sessions_per_month * cost_per_session)

where each session type has its own request count and its own average cost. Segment at least into light chat, retrieval-heavy questions, and agentic tasks. Track percentiles, not just means: the p50 user and the p99 user usually differ by an order of magnitude, and a flat-priced plan is sized by the tail.

A practical way to do it in a spreadsheet is a cohort table with three columns per segment: share of users, sessions per user per month, and cost per session. The blended cost per user is the SUMPRODUCT of the first two with the third. Then add a separate “heavy tail” row for the top one percent, because averages hide it.

Cost per outcome

Cost per request is an engineering metric; cost per outcome is the business metric. If the copilot resolves a ticket in 2.3 requests on average, cost per resolved ticket is 2.3 times the cost per request, divided by the resolution rate (some conversations need a human anyway, and their token cost is a pure expense). Failed or retried calls belong in the numerator. If you pay for tokens on a response the user abandons, or on a retry after a malformed JSON output, that is part of the true cost per outcome.

Agent Loops, Reasoning Blowup and Why Tails Eat Margin

The direct answer: an agent multiplies cost because each step re-sends the accumulated context as input, so total input tokens grow roughly with the square of the step count, while reasoning tokens inflate the output meter on every step. A task described as “one request” can be dozens of billed calls.

Figure 2: An illustrative eight-step agent run. Each step carries the full history, so total input tokens are far larger than the size of any single prompt.

Why input grows quadratically

A tool-using agent works by repeating a cycle: send the conversation so far, receive a model message that may include a tool call, run the tool, append the result, and call again. The provider is stateless between calls, so every call carries the whole history. If step 1 carries S tokens and every step appends A tokens of new content (tool output plus the model’s own message), then step k carries S + A(k-1) tokens, and the total over n steps is nS + A·n(n-1)/2.

With the illustrative numbers in Figure 2, S = 4,000 and A = 1,500, so eight steps send 8 × 4,000 + 1,500 × 28 = 74,000 input tokens. Add 400 output tokens per step, 3,200 in total. At Sonnet 5.5 list prices with no caching, that is 74,000 × $2 / 1,000,000 = $0.148 for input plus 3,200 × $10 / 1,000,000 = $0.032 for output, so $0.18 for one eight-step task. The simple chat request in the previous section cost $0.017, so this task costs about ten times as much even though a user might perceive both as “asking the assistant something”.

The input term is quadratic in n. Doubling the step count from 8 to 16 does not double input; it takes the formula to 16 × 4,000 + 1,500 × 120 = 244,000 tokens, which is 3.3 times as many. That is why loop caps, early-exit rules and context compaction are not nice-to-haves.

Caching changes the shape, not the direction

Because each step’s prompt begins with the previous step’s prompt, an agent is close to an ideal caching workload: the prefix is long and reused immediately. If we assume, illustratively, that 85 percent of input tokens across the run are cache reads and 15 percent are fresh or cache-write tokens billed at the 5-minute write rate of $2.50 per million (1.25 times base), the arithmetic is: 62,900 cache-read tokens at $0.10 per million is $0.0063, 11,100 write or fresh tokens at $2.50 per million is $0.0278, and output is still $0.032. Total about $0.066, down from $0.18, a 63 percent reduction.

Note what survived the optimization: the output line of $0.032 is now nearly half the run. Caching compresses the input side; it cannot touch generation. Agents that “think out loud” at every step therefore have a floor.

Real hit rates depend on cache TTL, whether tool results are inserted before or after the cached breakpoint, and whether anything in the prefix mutates. A timestamp or a reordered tool list in the system prompt resets the hit rate to zero. Verify the rate with usage fields rather than assuming it.

Reasoning tokens: the hidden multiplier

Suppose the model is allowed extended thinking and, on a hard step, spends 4,000 thinking tokens before a 500-token visible answer. Billed output is 4,500 tokens, so at $10 per million the step’s output costs $0.045 instead of $0.005, nine times the visible-answer cost. These token counts are illustrative; real thinking lengths vary by model, effort setting and problem difficulty, and you should measure them from the thinking_tokens usage field the provider returns.

This has three consequences. Budget caps matter: thinking budgets and effort settings are cost controls as much as quality controls. Routing matters: send easy turns to a non-reasoning or low-effort path, which is a topic we cover in LLM routing and model cascades. And measurement matters: if your dashboards count visible output only, your forecast will understate spend by the reasoning share, which can be most of the output bill.

One more detail from the Anthropic documentation: for newer models, prior turns’ thinking blocks stay in context and are billed as input on subsequent calls. In a multi-step agent that carries thinking blocks forward, reasoning tokens are billed twice in effect: once as output when generated and again as input on each later step that includes them (at the cache-read rate if cached). Account for that in the S and A parameters above.

The tail problem

Now put a distribution on it. Suppose, as an illustration, 80 percent of users run a few cheap chat turns, 18 percent run occasional agent tasks, and 2 percent are power users who delegate everything to the agent and run it all day. If a power user triggers 300 agent runs a month at the cached cost of $0.066 each, that is $19.80 a month; at the uncached $0.18 it is $54. Neither number is large in isolation, but under a $30 flat seat the uncached version is a loss on that user alone, and a 3x longer average loop would push even the cached version underwater.

The general rule: flat pricing is a bet on the shape of the distribution of consumption. When the marginal cost per unit of use is non-trivial and unbounded, either cap the use, price the use, or accept that your heaviest users subsidize nobody.

Gross Margin of an AI Feature: A Worked Example

The direct answer: AI feature gross margin is revenue per user minus the cost of goods sold per user, divided by revenue. COGS here is model inference plus the systems that serve it (retrieval, vector search, hosting, observability) plus variable payment and support costs. The calculation is easy; getting honest inputs is the hard part.

Figure 3: The margin stack. Sessions per user times tokens per session times price per token feeds model COGS, which is subtracted from revenue along with the non-model costs.

The formulas

In spreadsheet form, per user per month:

Revenue = seat_price
ModelCOGS = SUMPRODUCT(sessions_by_type, cost_per_session_by_type)
OtherCOGS = retrieval + hosting + observability + payment_fees + support_allocation
GrossProfit = Revenue - ModelCOGS - OtherCOGS
GrossMargin = GrossProfit / Revenue

Note OtherCOGS is not trivial. Embedding and vector search for retrieval, the log pipeline that stores prompts for evaluation, and the human review you run on low-confidence outputs can each add up. Many teams model only the provider invoice and are then surprised that the real margin is lower.

Illustrative P&L for a typical-seat user

Take a $30 per month seat for a support copilot. A typical user in this illustration runs 300 simple chat requests (at $0.017 uncached or $0.0113 cached) and 40 agent tasks (at $0.18 uncached or $0.066 cached) per month, plus $2.00 per user for retrieval, hosting and observability.

Line item	Naive (no caching)	With caching
Revenue	$30.00	$30.00
Chat: 300 requests	$5.10	$3.39
Agent: 40 tasks	$7.20	$2.64
Other COGS	$2.00	$2.00
Total COGS	$14.30	$8.03
Gross profit	$15.70	$21.97
Gross margin	52%	73%

Table: illustrative numbers built from the Sonnet 5.5 list prices cited above and invented usage volumes. Replace the usage with your own telemetry.

Caching moved margin from 52 to 73 percent in this scenario with no change in features or price. For comparison, gross margins above 70 percent are commonly cited as a benchmark for software, so the uncached design would look structurally weak against that yardstick. I label that benchmark as a widely repeated rule of thumb rather than a measured figure for this feature.

Now add the heavy user

Same $30 seat, same cached prices, but the user runs 400 agent tasks a month and 600 chat requests. Agent cost is 400 × $0.066 = $26.40, chat is 600 × $0.0113 = $6.78, other COGS $2.00, total $35.18, so gross profit is minus $5.18 and gross margin is minus 17 percent. A user who makes the feature visibly more valuable to themselves is the one losing you money. The blended margin across your base may still look fine, which is exactly why the mean hides the problem.

Sensitivity: which input moves margin most

Run a one-at-a-time sensitivity on the typical-user table. Doubling the agent steps per task (8 to 16) takes the uncached input from 74,000 to 244,000 tokens and the cached agent task cost from about $0.066 to about $0.176 on the same assumptions (the output line doubles too), which lifts agent COGS from $2.64 to roughly $7, and cuts margin from 73 to about 59 percent. Halving every token price, which the market has historically delivered over time, would lift margin to roughly 83 percent, because the $2 of non-model COGS does not fall. Swapping the model tier to a cheaper family changes cost by an order of magnitude but needs quality evidence.

The ordering to remember: usage volume and loop depth swing margin more than a 20 percent change in token price, which is why unit-economics work is mostly product and architecture work, not procurement.

What “AI-feature margin” should and should not include

Include inference, retrieval, serving infrastructure, evaluation traffic that runs in production, and any human-in-the-loop review that scales with usage. Exclude research and model-training costs unless you train your own models, but track fine-tuning and evaluation spend separately as R&D or as COGS depending on your accounting policy. Check with your finance team on classification. Investors and auditors treat these differently, and consistent treatment matters more than which side of the line you choose.

Pricing Models: Seat, Usage, Outcome

The direct answer: choose the pricing unit that tracks your cost driver and your customer’s perceived value. Seats are predictable but can be gamed by heavy use; usage-based pricing protects margin but adds bill anxiety; outcome-based pricing aligns with value but requires reliable attribution and absorbs your failure rate.

Seat pricing

Seats are easy to buy and easy to forecast. They work when the cost per user has a low variance, or when you can bound it with fair-use limits. The failure mode is the tail described above. Mitigations include included-credit pools, soft throttles on the most expensive actions, and routing heavy workloads to cheaper models after a threshold. A seat price should be set from the p80-to-p90 cost user, not the mean, if you want margin protection without capping.

Usage and credit pricing

Charging per token directly is simple for developers but opaque to business buyers. Many products abstract into credits or “actions” and set credit prices with a built-in margin over modeled cost. The trick is that the abstraction must be stable: if your credit price is fixed but your underlying cost per action drops, you gain margin; if the cost per action rises (a new reasoning default, a longer context), you lose margin unless you can re-denominate. Keep a version number on the credit-to-token conversion and log it.

Hybrid: seat plus metered overage

Most mature AI products converge on a base seat with an included allowance and metered overage. It gives finance a predictable floor and gives you a pressure valve for tail users. The allowance should be sized at the level where gross margin at the allowance is still above target, which is a back-solved calculation: allowance = (seat_price × (1 − target_margin) − OtherCOGS) / cost_per_unit.

For the example above, with a 70 percent target margin, a $30 seat and $2 other COGS, the model-cost budget is $30 × 0.30 − $2 = $7. If a blended unit (a cached agent task) costs $0.066, that is about 106 agent tasks. An allowance of 100 with overage beyond that keeps margin on target by construction.

Outcome-based pricing

Charging per resolved ticket, qualified lead or completed workflow aligns price with value. The arithmetic: price per outcome must exceed expected cost per outcome, where the denominator includes failures. If the cost per attempt is $0.20 and the success rate is 60 percent, the cost per successful outcome is $0.33, before human fallback and review. Outcome pricing exposes you to your own quality: a model regression that drops success by ten points raises cost per outcome by about 20 percent overnight with no change in price. It needs trajectory-level evaluation, covered in LLM agent evaluation.

Choosing

Model	Predictability for buyer	Margin protection	Best fit
Flat seat	High	Weak at the tail	Low-variance, bounded features
Seat plus allowance and overage	Medium to high	Strong	Most copilot-style products
Pure usage or credits	Low to medium	Strong	API and developer products
Outcome-based	Medium	Depends on success rate	Well-defined, attributable workflows

Optimization Levers and Their Quality Trade-offs

The direct answer: token cost optimization is a ranking problem. Apply the levers with zero quality risk first (caching, batching, output format), then the levers that need evaluation evidence (routing, context trimming, output caps), and measure each against a quality metric, not just a cost metric.

Figure 4: Cost levers and their quality risk. Caching and batch change the bill without changing outputs; the others trade some capability for savings.

Lever 1: prompt caching (low risk)

Put stable content first and variable content last, keep the prefix byte-identical, and use the longer TTL only when reuse justifies the higher write price. Anthropic documents that a five-minute write pays off after one read; with the one-hour write at twice base price it needs two. Watch the hit rate, not the configuration. Teams commonly discover that a per-request timestamp, a user name in the system prompt, or a non-deterministic JSON key order in the tool schema silently defeats the cache. Provider-side caching differs from application-level reuse, and semantic caching adds a different trade: it can skip the model call entirely but introduces a risk of returning an answer to a question that only looked similar. At the self-hosted layer, prefix caching in vLLM and SGLang plays the same role.

Lever 2: batch for anything not interactive (low risk, latency cost)

A 50 percent discount on both meters is large for evaluation runs, backfills, summarization pipelines and nightly enrichment. The cost is latency: batch jobs complete asynchronously, within a window the provider defines, so they cannot serve a user waiting on a screen. Move every non-interactive workload there by default.

Lever 3: model routing and cascades (medium risk)

Prices differ by up to two orders of magnitude within a vendor’s own family: on the Anthropic list, Haiku 5.5 at $0.10 input is one twentieth of Sonnet 5.5’s $2, and one hundredth of the $10 Fable 5.1 tier. Routing easy requests to the small model captures most of that gap. The risk is silent quality loss on the requests the router misjudges. Use a cascade with a confidence or verifier step, and evaluate the router on a labeled set that includes hard cases. The research basis (FrugalGPT, RouteLLM) and an implementation pattern are in our routing and cascades post. A gateway is the usual place to enforce routing and per-team budgets; see LLM gateways compared.

Lever 4: control output length and format (low to medium risk)

Since output is the high-priced meter, structured outputs, explicit length limits and “answer only” instructions pay back immediately. Ask for a JSON object rather than prose when a program consumes the result. Cap max_tokens at a sane ceiling. The risk is truncation: a cap that cuts off a JSON object produces a parse failure and a retry that costs more than the saving. Set caps from the observed p99 length, not from a guess.

Lever 5: context trimming and retrieval discipline (medium to high risk)

Input is cheaper than output but is usually the larger token count, and it compounds in agents. Retrieve fewer, better chunks (reranking), summarize or compact old turns, drop stale tool output, and avoid passing whole documents when a section will do. The risk is dropping the one fact the model needed. Measure answer quality with retrieval evaluation before and after; our posts on RAG retrieval evaluation and RAG chunking cover the metrics and tactics.

Lever 6: loop and reasoning governance (medium risk)

Cap steps per task, cap thinking budgets, and set a per-task dollar ceiling that terminates a runaway agent and escalates to a human. Use lower reasoning effort on routine steps. Because input grows quadratically with step count, bounding the loop is the single most effective control on agent cost. The trade-off is the tasks that legitimately need twenty steps; give them an explicit “extended” mode that is priced differently.

Lever 7: architecture changes (high effort)

Self-hosting open-weight models can beat API pricing at sustained high utilization, but you take on GPU capacity planning, ops and the idle-capacity cost. Cheaper architectures (linear-attention and recurrent models, discussed in RWKV vs Mamba vs linear attention) change the cost curve on long contexts but are a different quality bet. Do the arithmetic at your actual utilization before committing: a GPU priced per hour is cheap per token only if it is busy.

Instrument first

None of this works without per-request, per-feature, per-tenant token accounting. Tag every call with feature, tenant, model, cache-read tokens, thinking tokens and step index. Attribute cost on the same dimensions your finance team uses. Alert on the p99 cost per task, not only on total spend, so a loop bug shows up in hours instead of at invoice time.

Trade-offs, Gotchas, and What Goes Wrong

Token economics fails in predictable ways. The first is using the list price as the cost. The list price is the best case: it excludes cache misses, retries, failed generations, evaluation traffic, safety classifiers, embedding calls and the logging pipeline. Reconcile your model against the actual invoice monthly; a gap of more than a few percent means a meter is missing from your model.

The second is optimizing the mean. Caching and routing improve the average cost per request, but your risk lives in the tail: a single misconfigured agent with no step cap can burn through a daily budget in minutes. Per-task ceilings and per-tenant budgets at the gateway matter more than a few percentage points of average savings.

The third is optimizing cost without a quality gate. A cheaper model that is wrong more often creates retries, human escalations and churn, all of which cost more than the tokens saved. Pair every cost lever with an eval on the same task set, and track cost per successful outcome, not cost per call.

The fourth is price drift. Providers change prices, introduce new tiers, move caching defaults and deprecate models. A feature priced at a 70 percent margin on today’s list can be at 55 percent after a vendor change to the thinking default. Keep the price table in version control, re-read provider pages on a schedule, and keep your design portable across at least two vendors so you retain negotiating leverage.

The fifth is the context-window trap. Larger windows invite stuffing everything in. Even with cached reads at a small fraction of base price, 200K cached tokens re-read on every step of an agent is not free, and some models have size-tiered input prices, as the Haiku 5.5 threshold shows. Long context also degrades recall on many tasks, so the extra tokens may buy cost without quality.

The sixth is free-tier and trial abuse. Any feature with a free path and a variable cost needs per-account limits and anomaly detection from day one.

The seventh is privacy and compliance side effects. Data-residency options can carry a surcharge (Anthropic lists a 1.1 times multiplier for US-only inference on newer models), and logging full prompts for cost attribution creates a data-retention obligation. Log token counts and identifiers by default, and store content only where you need it and are allowed to.

Finally, remember that unit economics is not a one-off exercise. The model, the prompts, the user mix and the price list all move. Treat the cost model as living code with tests: a small script that replays a week of traces against the current price table and reports cost per feature, per percentile and per outcome.

Practical Recommendations

Start with measurement, then lock in the free wins, then run evaluations for the risky levers. In order:

Log tokens per request by meter (fresh input, cache read, cache write, thinking, visible output) with feature, tenant and step index. Without this you are guessing.
Build the spreadsheet: request classes, token columns, price cells, SUMPRODUCT, then session and user roll-ups with p50, p90 and p99 segments.
Fix the cache. Reorder prompts so stable content leads, remove volatile strings from the prefix, and verify the hit rate from usage fields.
Move all non-interactive work to batch endpoints.
Cap loops and thinking: set max steps, thinking budget and a per-task dollar ceiling with human escalation.
Route by difficulty, but only behind an eval gate and with a verifier on the cheap path.
Price with the tail in mind: seat plus allowance plus overage, with the allowance back-solved from your target margin and the p90 user’s cost.
Re-run the model monthly against the invoice and the current price pages; record the date of every price you use.

Checklist before you launch a priced AI feature:

Cost per request, session, user and outcome computed from real traces
p99 cost per task known, with a hard ceiling in code
Cache hit rate above your break-even threshold
Gross margin computed at p50 and p90 usage, including non-model COGS
Quality metric tracked next to cost for every optimization shipped
Price table versioned and dated; a second provider tested as a fallback

If these are in place, you can answer the question every CFO eventually asks: what does one more user cost us, and what happens to margin if usage doubles?

Financial and pricing figures in this article are for engineering and planning illustration, not investment, accounting or pricing advice. Verify current provider prices before relying on them.

Frequently Asked Questions

What is LLM token economics?

LLM token economics is the study of how token-based pricing turns into cost and margin for an AI product. It covers input, output, cached and reasoning token meters, how a user session maps to token counts, and how those costs compare with revenue per seat, per usage unit or per outcome. The aim is to predict gross margin and choose pricing and optimizations that protect it.

How do I calculate cost per token for an LLM?

Read the per-million-token prices for fresh input, cached input, cache writes and output from the provider’s pricing page, divide by one million for a per-token rate, and multiply by the tokens in each category on a real request. Sum the four lines to get cost per request. Include thinking tokens in the output line, and record the date of the price list you used.

Why do output tokens cost more than input tokens?

Output tokens are generated sequentially, one forward pass per token, which uses GPU time less efficiently than processing a whole prompt in parallel, so providers price them higher. In the 2026 lists cited here the ratio is about five to one, for example $2 input against $10 output per million tokens on Sonnet 5.5. The practical consequence is that shortening answers often saves more than trimming prompts.

How do reasoning tokens affect the bill?

Reasoning models generate internal thinking tokens that are billed at the output rate even when the response shows only a short answer. On Anthropic’s platform the thinking portion is reported in the usage object and counts toward max tokens. A short visible answer can therefore carry a multiple of its length in billed output. Track thinking tokens separately and cap budgets on routine steps.

What is a good gross margin for an AI feature?

There is no universal figure. Software is often held to a benchmark around 70 to 80 percent, but AI features carry real variable inference costs, so many land lower, especially with flat pricing and heavy users. Compute margin at p50 and p90 usage, include non-model COGS, and use the result to set an allowance, an overage rate or a model-routing policy that meets your target.

Should I use seat-based or usage-based pricing for an AI product?

Use seats when cost per user is low-variance or bounded, usage-based pricing when cost scales strongly with consumption, and a hybrid of seat plus allowance plus overage for most copilots. Outcome-based pricing fits workflows with clear attribution but exposes you to your own failure rate. Pick the unit that tracks your cost driver, then back-solve the allowance from your target gross margin.