LLM Prompt Caching Economics: Explicit vs Implicit, TTLs and Break-Even Math

Most teams that turn on LLM prompt caching assume it can only save money. It cannot. Every major hosted API now charges a premium to write a prefix into the cache, and a cache that expires before the next request arrives turns a 1.0x input bill into a 1.25x or 2x bill. Whether caching pays depends on three numbers: the write multiplier, the read multiplier, and how often the same prefix is requested inside its time to live (TTL).

This matters more in 2026 than it did a year ago because agent loops, long system prompts and large tool catalogs have pushed typical prompts past 20,000 tokens, and input tokens dominate the invoice. The providers have also diverged: Anthropic bills explicit writes with a choice of TTLs, OpenAI added explicit breakpoints and a 12-hour tier for its newest models, and Google bills explicit caches by storage time.

You will leave with the verified mechanics of each provider, closed-form break-even formulas, an arrival-rate model that tells you which TTL to buy, and a code example that logs real cache usage.

What this covers: the economics of explicit versus implicit caching, a provider-by-provider comparison, break-even math with worked numbers, TTL selection, cache-busting failure modes, and a rollout checklist.

Context and Background

Prompt caching stores the intermediate attention state, the key-value (KV) tensors, for a prompt prefix so the model does not recompute it. The provider is selling you back a saving that exists on its GPUs. If you want the inference-engine view of that mechanism, our deep dive on prefix caching with RadixAttention versus vLLM automatic prefix caching covers how self-hosted servers do it. This post is deliberately about the hosted-API side: what the price list and the TTL rules mean for your invoice.

Three facts frame everything below. First, caching is prefix-based: a hit requires that the leading tokens of the new request match a stored prefix exactly, so one changed token early in the prompt invalidates everything after it. Second, the saving is on input tokens only. Output tokens are never discounted, so a workload that generates long answers from short prompts gets little out of caching. Third, the three providers disagree on who decides where the cache boundary sits, and that disagreement is the explicit-versus-implicit question.

Explicit versus implicit, defined

In explicit caching the developer marks the boundary. Anthropic’s cache_control markers and OpenAI’s prompt_cache_breakpoint (on GPT-5.6 and later) are examples, as is Gemini’s separate cache object created through caches.create. You pay for exactly what you mark, and you can leave volatile content outside the cached region.

In implicit caching the provider picks the boundary and applies the discount automatically. OpenAI’s earlier models work this way, Gemini 2.5 and newer have it on by default, and OpenAI’s GPT-5.6 and later offer it as a mode. You write no code, but you also cannot guarantee a hit, and on some providers you cannot control the TTL.

Anthropic blurs the line. Its automatic caching is a single top-level cache_control field that moves the breakpoint to the last cacheable block as a conversation grows. It is still billed as a cache write at the write multiplier, so it is closer to “explicit with convenience” than to free implicit caching.

What the pricing pages say, as retrieved on 2026-11-07

I retrieved the official pages for Anthropic prompt caching, OpenAI prompt caching, and the Gemini API context caching and pricing pages on the day of writing. Model names and prices below come from those pages. They change often, so treat every dollar figure as a snapshot and re-check before committing a budget. Where I use a multiplier rather than a dollar price, it is the form most likely to survive the next price change.

The Reference Model: One Formula, Three Providers

Prompt caching pays off when the expected cost per request, write multiplier times the miss probability plus read multiplier times the hit probability, falls below 1.0 of the uncached input price. With a write multiplier w and read multiplier r, caching beats no caching once you reuse a prefix more than (w minus r) divided by (1 minus r) times.

Figure 1: The lifecycle of a cached prefix. A hit pays the read multiplier and refreshes the TTL on Anthropic and OpenAI. A miss pays the write multiplier and starts a new entry.

Figure 1 shows the loop every provider implements. The request arrives, the provider computes a hash of the leading tokens at each candidate boundary, and it looks for a stored entry. A hit charges the read multiplier on the matched tokens. A miss charges the write multiplier on the tokens that become cache entries, then stores them. Tokens after the last boundary are charged at the ordinary rate. The details that differ are who picks the boundary, what a write costs, and how long an entry lives.

The break-even formula

Let P be the prefix length in tokens, p the ordinary input price per token, w the write multiplier and r the read multiplier. Suppose the prefix is sent in N requests, all inside the TTL.

Uncached cost is N times P times p. Cached cost is P times p times (w + (N minus 1) times r), since the first request writes and the remaining N minus 1 read. Caching wins when w + (N minus 1) times r is less than N, which rearranges to:

N is greater than (w minus r) divided by (1 minus r).

Provider documentation gives a concrete instance. OpenAI’s page states that with the 0.1x read rate and a 30-minute write, writing a prefix once and fully reusing it once costs 1.35x its ordinary input cost, against 2x for processing it twice without caching, and that one write plus nine reads costs 2.15x against 10x. Those figures match the formula with w of 1.25 and r of 0.1: 1.25 plus 0.1 is 1.35, and 1.25 plus 9 times 0.1 is 2.15.

Break-even request counts by tier

Applying the formula to the multipliers the vendors publish gives this table. “Requests” is the minimum number of requests inside one TTL window, including the one that pays the write, for caching to cost less than not caching.

Tier	Write (w)	Read (r)	Break-even N	Smallest whole N	Cost at N=2	Cost at N=10
Anthropic 5-minute	1.25x	0.10x	1.28	2	1.35x	2.15x
Anthropic 1-hour	2.0x	0.10x	2.11	3	2.10x (a loss)	2.90x
Anthropic 5-minute, 0.05x read models	1.25x	0.05x	1.26	2	1.30x	1.70x
Anthropic 1-hour, 0.05x read models	2.0x	0.05x	2.05	3	2.05x (a loss)	2.45x
OpenAI 30-minute (GPT-5.6+)	1.25x	0.10x	1.28	2	1.35x	2.15x
OpenAI 12-hour (GPT-5.6+)	2.5x	0.10x	2.67	3	2.60x (a loss)	3.40x

The pattern is clear. A short-TTL tier pays for itself on the second request. A long-TTL tier needs a third request, and if you only reuse the prefix once it costs about as much as sending it twice uncached, plus a bit more. The long tiers are not a discount; they are insurance that lets a later request still find the entry.

Why the read multiplier is the lever

Because the break-even count barely moves with r, the read multiplier mostly sets the size of the long-run saving rather than the threshold. At 0.1x, a perfectly reused prefix costs 10 percent of normal input in the limit. At 0.05x, as Anthropic lists for Claude Opus 5.5 and Claude Sonnet 5.5, and OpenAI lists for GPT-6.1 Sol and GPT-6.1 Terra, it costs 5 percent. Anthropic’s footnote also lists 0.025x for Claude Fable 5.1 and Claude Mythos 5.1. That halving matters for agents that replay 100,000-token contexts hundreds of times, but it does not change when caching starts to pay.

Provider Mechanics, Verified

Each provider exposes a different surface. This section states what the retrieved documentation says, and flags what it does not.

Anthropic: explicit breakpoints, two TTLs

Anthropic offers two ways to enable caching on the Messages API. Automatic caching adds a single top-level cache_control field, and the system applies the breakpoint to the last cacheable block, moving it forward as the conversation grows. Explicit caching places cache_control on individual content blocks, with up to four breakpoints per request. When both are used, the automatic breakpoint consumes one of the four slots.

The default TTL is 5 minutes, and the documentation states the cache is refreshed at no additional cost each time the content is used. The lifetime is measured from the start of the request that writes or reads the entry, not from the end of its response, so a four-minute stream leaves roughly one minute for a follow-up. A 1-hour TTL is available by adding "ttl": "1h" to the marker.

Published multipliers: a 5-minute write is 1.25x base input, a 1-hour write is 2x, and a read is 0.1x with per-model exceptions at 0.05x and 0.025x as noted above. These multipliers stack with the Batch API discount, long-context pricing and data residency pricing, according to the same page. As a dollar example from the table I retrieved, Claude Sonnet 5.5 lists input at $2 per million tokens, a 5-minute write at $2.50, a 1-hour write at $4 and a hit at $0.10.

Minimum cacheable lengths vary by model. The page lists 512 tokens for the newest models including Claude Opus 5.5, Sonnet 5.5 and Haiku 5.5, 1,024 for several others, 2,048 for Claude Opus 4.7, and 4,096 for Claude Opus 4.6, Opus 4.5 and Haiku 4.5. A prompt below the minimum is processed without caching and no error is returned, which is why the usage fields matter.

The prefix is built in the order tools, then system, then messages. Anthropic’s lookback examines at most 20 positions per breakpoint, counting the breakpoint itself, looking for entries earlier requests wrote. If a conversation grows more than 20 blocks past the last write, an automatic breakpoint can silently miss, and the fix is a second explicit breakpoint closer to the earlier write.

OpenAI: implicit by default, explicit from GPT-5.6

OpenAI’s page says prompt caching is enabled by default for supported models. For GPT-5.6 and later it offers two modes, set through prompt_cache_options.mode. In implicit mode the system places a breakpoint at the end of the latest eligible message. In explicit mode you mark breakpoints with prompt_cache_breakpoint, and if you place none, the request does not use caching or create writes. A request can create up to four cache writes.

Pricing for GPT-5.6 and later, per the page: a write for the default "30m" TTL costs 1.25x the uncached input rate, a "12h" write costs 2.5x, and reads cost 0.1x on most models and 0.05x on GPT-6.1 Sol and GPT-6.1 Terra regardless of lifetime. The minimum cacheable prefix is 1,024 visible input tokens for GPT-5.6 and later. Earlier models use implicit caching only, report cached tokens rounded down to a multiple of 128, and have no additional write charge, with cached-input rates that vary by model.

Earlier models also use a prompt_cache_retention parameter with in_memory (entries typically active for 5 to 10 minutes of inactivity, up to one hour) and 24h on a listed set of models. For models before GPT-5.6, a stable prompt_cache_key helps route related requests to the same machine, and the documentation advises staying near 15 requests per minute per key because higher traffic can overflow to other machines. On GPT-5.6 and later, routing is automatic and the key is optional, though it still separates cache accounting by customer.

Gemini: implicit by default, explicit by storage-hour

Google’s documentation says implicit caching is on by default for Gemini 2.5 and newer, passes savings on automatically, and carries no cost-saving guarantee. Minimum input sizes in the table I retrieved are 4,096 tokens for Gemini 3.8 Flash, Gemini 3.6 Flash and Gemini 3.1 Pro Preview, and 2,048 for Gemini 2.5 Flash and Pro. The page recommends placing large common content first and sending similar-prefix requests close together.

Explicit caching is in beta on the v1beta endpoints, supported on Gemini 3 and 2.5 models, and is not supported from Gemini 4 Argon onward. You create a cache object with caches.create, give it a TTL (the default is one hour when unset, with no stated minimum or maximum bounds), and reference it on later calls via cached_content. Billing has three parts: cached tokens at a reduced rate when used, storage for the TTL duration, and ordinary charges for uncached input and output.

On the pricing page, Gemini 3.8 Flash lists paid-tier input at $0.75 per million tokens through December 31, 2026 (rising to $1.50 from January 1, 2027), a context-caching read price of $0.075 (rising to $0.15), and storage at $0.50 per million tokens per hour (rising to $1.00). The scheduled increases matter if you build a long-lived budget. One more detail from the docs: implicit caching does not apply when a request uses an explicit cache, and extra input alongside the cache is billed at the standard input rate.

For agent loops on Gemini 4 and newer, the documentation notes that the full previous request is cached only if the new request adds at least 2,050 tokens; for smaller additions the last 2,050 tokens of the previous request are re-processed at the standard input rate. That is an unusual rule, and it makes very short tool-result turns more expensive than their size suggests.

Break-Even Math Under Real Traffic

The N-request formula assumes every request lands inside the TTL. Production traffic does not behave that way. Requests arrive at irregular intervals, and each gap either falls inside the TTL, in which case you read and the entry is refreshed, or outside it, in which case you pay a fresh write. The question that decides your bill is therefore the probability that the next request arrives before the entry expires.

A steady-state model with refresh on hit

Assume a single prefix, refresh on every hit (documented for Anthropic’s cache and for OpenAI’s 30-minute reads), and let q be the probability that the gap to the next request is shorter than the TTL. Expected cost per request, as a multiple of the uncached input cost, is q times r plus (1 minus q) times w. Caching beats no caching when that is below 1, which gives a clean threshold:

q must exceed (w minus 1) divided by (w minus r).

For a 5-minute write at 1.25x and a 0.1x read, the threshold is 0.25 divided by 1.15, about 0.22. For a 1-hour write at 2x, it is 1.0 divided by 1.9, about 0.53. In words: a 5-minute cache only needs about one in five follow-ups to arrive in time; a 1-hour cache needs about one in two.

Figure 2: One prefix under irregular traffic. Gaps inside the TTL refresh the entry at the read price; a gap beyond the TTL forces a rewrite at the write price.

What the threshold means in minutes

If requests arrive as a Poisson process with a mean gap of G minutes, then q equals 1 minus e to the power of minus T over G, where T is the TTL in minutes. I computed the expected cost per request for four tiers. This is an illustrative model, not a measurement: real traffic is burstier than Poisson, which helps a short TTL because bursts cluster hits, and it ignores the way a long-minimum-lifetime tier may keep entries longer than promised.

Mean gap between requests	Anthropic 5m (1.25x/0.1x)	Anthropic 1h (2x/0.1x)	OpenAI 30m (1.25x/0.1x)	OpenAI 12h (2.5x/0.1x)
1 minute	0.11	0.10	0.10	0.10
5 minutes	0.52	0.10	0.10	0.10
10 minutes	0.80	0.11	0.16	0.10
20 minutes	1.00	0.20	0.36	0.10
30 minutes	1.07	0.36	0.52	0.10
60 minutes	1.16	0.80	0.80	0.10
90 minutes	1.19	1.08	0.92	0.10
120 minutes	1.20	1.25	1.00	0.11

Values are expected cost per request as a multiple of uncached input. Anything below 1.00 saves money. Illustrative Poisson model, single prefix, refresh on hit.

Three conclusions follow. A 5-minute tier stops paying for itself once the mean gap passes about 20 minutes, and by 30 minutes it costs 7 percent more than not caching at all. A 1-hour tier holds its advantage until the mean gap is roughly 100 minutes. And the 12-hour OpenAI tier looks almost free in this model, but only because the model assumes entries live the full window; the documentation says 12 hours is a minimum lifetime that governs read-pricing eligibility and does not guarantee that every request reuses stored KV tensors. Treat the 12-hour row as an upper bound on the benefit.

Choosing between a short and a long tier

The 1-hour tier costs 0.75x of base input more per write than the 5-minute tier (2.0 versus 1.25). It earns that back only on the requests that the 5-minute tier would have missed. For a deterministic cadence, the rule is simple: if the interval between requests is under 5 minutes, use the 5-minute tier and never pay the 2x; if it is between 5 and 60 minutes, the 1-hour tier is the only one that hits, so the choice is 1-hour versus no caching.

Anthropic’s own guidance says the same. A prompt sent more often than every 5 minutes should stay on the 5-minute cache because it refreshes for free, and the 1-hour cache suits side-agents that run longer than 5 minutes or chat users who may pause before replying. The documentation also lists a non-price benefit: cache hits are not deducted against your rate limit, which can matter more than the discount at high volume.

A worked example for an agent workload

Take a support agent with a 40,000-token prefix (tools, policy documents, few-shot examples), 500 new input tokens per request and 300 output tokens, run 100 times inside the TTL window. Use the Claude Sonnet 5.5 prices from the table I retrieved: $2 per million input, $10 per million output, $2.50 per million for a 5-minute write and $0.10 per million for a read.

Uncached, the prefix costs 40,000 times $2 per million, or $0.08 per request. With the other input and output the total is $0.084 per request, so 100 requests cost $8.40. Cached, one request pays the write at $0.10, and the other 99 pay $0.004 each to read, so the prefix costs about $0.496 in total. Adding the unchanged $0.004 per request for the new input and output tokens gives about $0.90 for the run. That is a reduction of roughly 89 percent. If the read price were 0.1x rather than 0.05x, the same run would cost about $1.29.

Notice where the remaining cost sits. After caching, the output tokens and the uncached suffix dominate. At that point, further savings come from shortening the answers, not from more caching.

Gemini explicit caching: a storage-time break-even

Gemini’s explicit cache has no write multiplier in the pages I retrieved, but it charges for storage time. I did not find a statement of how cache creation itself is billed, so the following assumes creation is processed at the ordinary input rate. That is an assumption, and you should confirm it on your billing console before relying on it.

Per million cached tokens, with p as the ordinary input price, c as the cached read price, s as the storage price per hour and H as the TTL in hours, explicit caching costs p (creation) plus N times c plus s times H over N requests, against N times p without it. Caching wins when N exceeds (p plus s times H) divided by (p minus c).

With the Gemini 3.8 Flash figures through December 2026 ($0.75, $0.075, $0.50 per hour), a one-hour cache breaks even at 1.85 requests, so two. A six-hour cache needs 5.6 requests, so six. A 24-hour cache needs about 18.9, so 19 requests that day. After the January 1, 2027 repricing in the pages I retrieved, p becomes $1.50, c becomes $0.15 and s becomes $1.00, and the one-hour break-even is 2.5 divided by 1.35, about 1.85 again, because every price doubled. The ratio matters more than the level.

A 100,000-token prefix queried 20 times in one hour, with 400 new tokens and 300 output tokens each, costs about $1.53 uncached and about $0.30 with a one-hour explicit cache under these prices. The saving looks large, but the same workload on implicit caching might achieve most of it for free if requests are close together, which is why Google’s page frames implicit caching as having no guarantee rather than no value.

Choosing Explicit or Implicit: A Decision Framework

Figure 3: A TTL and mode decision flow. Traffic gap and prefix stability decide the answer before the provider does.

The decision in Figure 3 starts with the prefix itself. If it is below the provider’s minimum cacheable length, no mode helps; the request is processed uncached with no error. If it is stable and large, the next question is the gap between requests. Gaps under five minutes favor the shortest explicit tier or an implicit default. Gaps between five and sixty minutes favor a long tier. Gaps over an hour favor either a long-lifetime tier priced for rare reuse or no caching.

Where implicit wins

Implicit caching wins when you cannot predict reuse and the cost of a wrong guess matters. OpenAI’s earlier models and Gemini’s implicit mode carry no write charge according to the retrieved pages, so a miss costs nothing extra. You trade control for safety. That is the right trade for chat products with variable traffic, internal tools, and anything where an engineer will not tune breakpoints.

Where explicit wins

Explicit caching wins when you know the structure of your prompt and the volume is high. Three conditions favor it: a large stable prefix reused by many requests; volatile content you want kept out of the write (OpenAI’s documentation notes that explicit-only mode leaves content after the last breakpoint at the uncached rate with no write charge); and a need for a longer TTL than the default. The cost is that wrong breakpoints can lose money: a breakpoint on content that changes every request produces a write each time and a read never.

Anthropic’s docs show this exact mistake. A timestamp inside the last block means each request has a different prefix hash, the lookback finds nothing, and you pay the write multiplier on every call. The remedy is to put the breakpoint on the last block that is identical across requests, which in a prompt with a varying suffix is the end of the static prefix.

Implementation: Measuring Cache Behavior in Code

You cannot tune what you do not log. Every provider returns cache counters in the response, and a small wrapper that records them gives you the realized read ratio, which is the only number that tells you whether your model of the traffic was right.

The following Python sketch uses the Anthropic and OpenAI shapes from the retrieved documentation. The model identifiers are placeholders taken from the documentation examples; substitute your own.

import anthropic
from openai import OpenAI

STATIC_PREFIX = open("policy_and_tools.txt").read()  # > provider minimum

def anthropic_call(question: str, ttl: str = "5m"):
    client = anthropic.Anthropic()
    cc = {"type": "ephemeral"}
    if ttl == "1h":
        cc["ttl"] = "1h"
    resp = client.messages.create(
        model="claude-sonnet-5-5",          # placeholder, check docs
        max_tokens=512,
        system=[{"type": "text", "text": STATIC_PREFIX,
                 "cache_control": cc}],     # breakpoint at end of static prefix
        messages=[{"role": "user", "content": question}],
    )
    u = resp.usage
    return resp, {
        "write": u.cache_creation_input_tokens,
        "read": u.cache_read_input_tokens,
        "uncached": u.input_tokens,         # tokens after the last breakpoint
    }

def openai_call(question: str, ttl: str = "30m"):
    client = OpenAI()
    resp = client.responses.create(
        model="gpt-5.6",                    # placeholder, check docs
        prompt_cache_options={"mode": "explicit", "ttl": ttl},
        input=[
            {"role": "developer", "content": [{
                "type": "input_text", "text": STATIC_PREFIX,
                "prompt_cache_breakpoint": {"mode": "explicit"}}]},
            {"role": "user", "content": question},   # after breakpoint
        ],
    )
    d = resp.usage.input_tokens_details
    return resp, {"read": d.cached_tokens,
                  "write": getattr(d, "cache_write_tokens", 0)}

def expected_cost_per_req(w, r, q):
    """Expected multiple of uncached input cost, hit probability q."""
    return q * r + (1 - q) * w

def break_even_q(w, r):
    return (w - 1) / (w - r)

Two details in that code come straight from the documentation. On Anthropic, input_tokens counts only the tokens after the last breakpoint, so total input is the sum of read, write and input_tokens. On OpenAI, the explicit-mode request leaves the user question after the breakpoint so it incurs no write. The Gemini equivalent creates a cache with client.caches.create(model=..., config=types.CreateCachedContentConfig(system_instruction=..., contents=[...], ttl="3600s")) and passes cached_content=cache.name in the generate config.

Pre-warming and concurrency

Anthropic documents a pre-warming request: set max_tokens to 0 and the API writes the cache at your breakpoints and returns immediately with an empty content array. Use the same thinking and effort settings as the real request, because those values are rendered into the prompt and a mismatch means the warmed entry does not match. The documentation also warns that a cache entry only becomes available after the first response starts, so fan-out requests fired in parallel all miss. Send one, wait for the first token, then send the rest.

Figure 4: Order the prompt from most stable to least stable. A change at any layer invalidates that layer and everything after it.

Figure 4 encodes the rule that drives most cache failures. Tools come first, then the system prompt, then static context, then the volatile suffix. Anthropic’s invalidation table says that changing tool definitions invalidates the tools, system and messages caches, while changing tool_choice or adding images affects only the message blocks. OpenAI’s page makes the equivalent point for its settings: a different model, different tool schemas or ordering, parallel_tool_calls, a structured-output format, reasoning.effort, text.verbosity and compaction can all change the rendered prefix. The practical consequence is that a feature flag toggled per user can quietly fragment your cache into dozens of small ones.

Keeping tool lists stable

Agents that load different tools per request are the most common source of cache fragmentation. Both vendors now document append-only patterns. OpenAI recommends keeping tool definitions, ordering and schemas fixed, disabling tools with tool_choice: "none" rather than removing them, and restricting callable tools with allowed_tools while the supplied list stays stable. Anthropic describes mid-conversation system messages, and for models that support it a beta header for tool changes made inside a message, so the top-level tools array and the cached prefix stay unchanged. Both patterns make the same trade: a slightly larger prompt in exchange for a prefix that never changes.

Trade-offs, Gotchas, and What Goes Wrong

Caching is a bet on traffic shape, and the failure modes are mostly about losing that bet quietly.

Paying the write on every request. The costliest bug is a breakpoint placed on volatile content. On a 40,000-token prefix at a 1.25x write rate, a prompt that never hits costs 25 percent more than not caching, and nothing in your error logs shows it. The only signal is a read counter that stays at zero while the write counter climbs. Alert on the ratio of read tokens to write tokens, not on errors.

Long TTLs that never get reused. A 1-hour or 12-hour write at 2x to 2.5x loses money when a prefix is read once. Per-user caches are the classic case: thousands of users each pay for a long-lived entry that the user never returns to. Long tiers belong on shared prefixes (a system prompt, a policy corpus), not on per-user history that goes cold after one session.

Silent non-caching below the minimum. Anthropic returns no error when a prompt falls below the model’s minimum, and the minimums run from 512 to 4,096 tokens depending on the model. If you swap models for cost reasons, a prefix that cached on one may not cache on another. Check the usage fields after every model change.

Lookback gaps in long agent runs. Anthropic’s 20-position lookback means a conversation that adds 20 or more blocks between requests can miss an automatic breakpoint. Parallel tool calls are somewhat protected, since consecutive tool-use blocks and consecutive tool-result blocks each count as one position, but a long sequence of distinct steps is not. Add a second breakpoint to anchor an earlier write.

Concurrency races. Parallel fan-out hits an empty cache because the entry is only available after the first response starts. A burst of 50 identical requests can therefore pay 50 writes. Warm first, or serialize the first call.

Thinking and settings drift. Anthropic documents that changing the thinking configuration or the effort value invalidates the message cache, and on some models also the tool and system caches. OpenAI documents similar sensitivity to reasoning.effort, and provides a configuration-update item on supported GPT-6 models so you can change effort without rewriting the prefix. If your product lets users pick a reasoning level per request, you may be paying for multiple cache families.

Privacy and isolation. Caches are scoped. OpenAI states caches are not shared across organizations or across regional processing boundaries, and recommends separate prompt_cache_key values to prevent cache-hit probing across users, where an attacker submits candidate prompts and watches for hits. Retention is also a compliance variable: OpenAI notes that the default retention for models that support both modes depends on Zero Data Retention status. If your data policy limits storage duration, pick the TTL accordingly.

Price drift. Gemini’s own pricing page schedules increases for input, cache reads and storage on January 1, 2027. Cache economics depend on ratios, and those ratios survived the doubling above, but a change in the ratio between storage and input would move the break-even point. Re-run the formulas whenever a price page changes.

Assuming implicit means free. Implicit caching has no write premium on the models described in the retrieved pages, but it also has no guarantee. Gemini’s page says so directly. If your budget needs a guaranteed discount, implicit caching cannot give you one; only an explicit cache with enough requests can.

Practical Recommendations

Start with measurement, not configuration. Turn on usage logging for the cache counters on every provider you use, run a week of production traffic, and compute the realized read ratio, meaning cached tokens divided by total input tokens. Everything else follows from that number.

Then decide per prefix, not per application. A single product often has three prefixes with three different shapes: a global system prompt reused thousands of times an hour, a per-tenant knowledge block reused dozens of times, and a per-session conversation that grows. The global one belongs on a short TTL with a stable breakpoint. The tenant block may justify a long tier if tenants return within the hour. The session history is where automatic or implicit caching earns its keep, because the breakpoint should move with the conversation.

Use the formulas in this post to set thresholds. If your measured probability of a follow-up inside the TTL is below about 0.22 for a 5-minute write, or about 0.53 for a 1-hour write at 2x, turn caching off for that prefix. For Gemini explicit caches, divide the storage cost by the saving per request to find the minimum number of requests per TTL.

Checklist:

Confirm the prefix exceeds the model’s minimum cacheable length, and recheck after any model change.
Order the prompt: tools, system, static context, then volatile content such as timestamps and user identifiers.
Place breakpoints at the end of the last block that is identical across requests.
Keep tool definitions, ordering and schemas fixed; disable rather than remove.
Choose the shortest TTL that covers the real gap between requests; reserve long tiers for shared prefixes.
Warm the cache once before parallel fan-out.
Alert on a high write-to-read ratio and on a read counter stuck at zero.
Use separate cache keys where tenants must not share cache accounting.
Re-run the break-even math whenever a provider changes prices or TTL rules.

Frequently Asked Questions

What is LLM prompt caching and how does it reduce cost?

LLM prompt caching stores the key-value state the model computed for a prompt prefix, so a later request that begins with the same tokens skips that computation. The provider bills reused tokens at a read rate, often one tenth of the normal input price, and sometimes lower. You only save on the repeated prefix, not on new input or output tokens. A write premium of 1.25x to 2.5x applies on some providers, so reuse count and timing decide the net result.

Is implicit or explicit prompt caching cheaper?

Neither is cheaper in general. Implicit caching has no write premium on the models described in the retrieved Gemini and earlier OpenAI documentation, so a miss costs nothing extra, but hits are not guaranteed. Explicit caching charges a write premium on Anthropic and on OpenAI GPT-5.6 and later, and storage-time fees on Gemini, but lets you control what is cached. Explicit wins with high-volume stable prefixes. Implicit wins with unpredictable traffic.

How long does a prompt cache last?

It depends on the provider and tier. Anthropic defaults to 5 minutes, refreshed at no cost on each use, with a 1-hour option at 2x write price. OpenAI GPT-5.6 and later offers a 30-minute default and a 12-hour minimum lifetime in the Responses API, while earlier models use in_memory or 24h retention. Gemini explicit caches default to a one-hour TTL when unset, and you can update the TTL on an existing cache.

How many requests do I need before caching pays off?

With a 1.25x write and 0.1x read, two requests inside the TTL are enough, since one write plus one read costs 1.35x against 2x uncached. With a 2x write, such as Anthropic’s 1-hour tier, you need three. With OpenAI’s 2.5x 12-hour write, you also need three. The general rule is that N must exceed (w minus r) divided by (1 minus r), where w and r are your write and read multipliers.

Why is my cache hit rate zero?

Common causes: the prefix is shorter than the model’s minimum cacheable length; something volatile, such as a timestamp, sits before the breakpoint; tool definitions or settings like reasoning effort changed; requests were sent in parallel before the first response started; or the TTL expired between requests. On Anthropic, check whether both cache_creation_input_tokens and cache_read_input_tokens are zero. Both vendors also offer cache diagnostics tools that report where a prefix diverged.

Do cached tokens count against rate limits?

On Anthropic, the documentation lists improved rate limit utilization as a reason to use the 1-hour cache, because cache hits are not deducted against your rate limit. Check your own account’s rate-limit documentation for specifics, since limits differ by provider, model and tier. Note too that Anthropic’s guidance says total input is the sum of read, write and uncached tokens, so cost accounting and rate limit accounting are not the same thing.

LLM API Prompt Caching Economics: Explicit vs Implicit Caching, TTLs and Break-Even Math Across Providers (2026)