Diagnosing Claude API Prompt Cache Misses — How to Read the usage Field

Cause #2: cache_control is in the wrong place

Prompt caching treats every block up to and including the one with cache_control as a single cache key. The cache_control marker has to sit on the last block of the static prefix.

# ❌ Wrong: cache_control sits after a dynamic block
system=[
    {"type": "text", "text": SYSTEM_INSTRUCTION},
    {"type": "text", "text": f"Today is {today}"},  # changes daily
    {"type": "text", "text": "Output format: ...",
     "cache_control": {"type": "ephemeral"}},
]

That layout pulls today's date into the cache key, so the cache rotates every midnight and you never get a hit. Move dynamic content after the cached block.

# ✅ Correct: dynamic block comes after the cached prefix
system=[
    {"type": "text", "text": SYSTEM_INSTRUCTION},
    {"type": "text", "text": "Output format: ...",
     "cache_control": {"type": "ephemeral"}},
    {"type": "text", "text": f"Today is {today}"},  # outside the cache
]

The same rule applies to tools. Putting cache_control on the last entry of your tools array caches every tool definition above it as one prefix.

Cause #3: The TTL has expired

The default cache TTL is 5 minutes. If five minutes elapse without a request that hits the same prefix, the cache is gone and the next request pays the write cost again. For low-frequency batch jobs or chatbots where users idle for ten minutes between turns, the cache rarely survives long enough to pay back its write premium.

The 1-hour TTL that became generally available in 2026 is the right tool for these workloads. Writes cost 2× standard instead of 1.25×, but the longer survival makes it cheaper overall when access is sparse.

system=[
    {
        "type": "text",
        "text": LARGE_SYSTEM_PROMPT,
        "cache_control": {"type": "ephemeral", "ttl": "1h"},
    }
]

Log the ratio of cache_read_input_tokens to cache_creation_input_tokens over a full day before and after the change so you have evidence the switch was worth the higher write premium.

Cause #4: A dynamic value sneaks into the cached prefix

This is the bug I keep making. A "static" prefix turns out to contain something that quietly drifts on every request.

Patterns I've seen in production code:

• A timestamp like Current time: 2026-04-28T09:00:34Z injected at the top of the system prompt
• A user ID embedded in the system prompt to "remind" Claude who it's talking to
• A per-session tracking UUID baked into the cached prefix
• Inconsistent line endings (\n vs. \r\n) across environments

The fastest way to catch these is to hash the cached portion before you send it.

import hashlib
 
def cache_key_hash(text: str) -> str:
    return hashlib.sha256(text.encode("utf-8")).hexdigest()[:16]
 
prefix = "".join(b["text"] for b in system_blocks_until_cache_control)
print(f"cache prefix hash: {cache_key_hash(prefix)}")

If the hash changes between requests that ought to be identical, the cause is in your prefix construction. Diff two consecutive prefixes side by side; the offending field tends to be obvious. This single trick saved me half a day chasing a phantom Anthropic-side issue when the real culprit was a request timestamp my own code was adding.

Cause #5: Tool ordering isn't deterministic

If you cache a tools array and still see misses, look at the ordering. Claude compares cache keys as raw bytes, so the same set of tools in a different order is a different cache.

When you build the array from a Python dict whose insertion order depends on runtime conditions, the array shuffles between requests and the cache rotates with it. Pin the ordering with an explicit list.

TOOL_ORDER = ["search_docs", "execute_sql", "send_email"]
 
def build_tools(enabled: set[str]) -> list[dict]:
    # Always emit tools in a fixed order
    return [TOOL_DEFINITIONS[name] for name in TOOL_ORDER if name in enabled]

If different users genuinely have different tool sets, the cache hit rate has a hard ceiling no matter what you do. The realistic design is to split tools into "common" and "user-specific" blocks, attach cache_control only to the common block, and append the per-user tools after it.

A diagnostic script: three calls in a row

Here's the small script I keep around for triage. It fires the same prompt three times and prints how usage evolves.

import anthropic
from anthropic.types import TextBlockParam
 
client = anthropic.Anthropic()
 
SYSTEM_PROMPT = open("system_prompt.txt").read()  # must be >= 1,024 tokens
 
def diagnose():
    for i in range(3):
        resp = client.messages.create(
            model="claude-sonnet-4-6",
            max_tokens=64,
            system=[
                TextBlockParam(
                    type="text",
                    text=SYSTEM_PROMPT,
                    cache_control={"type": "ephemeral"},
                )
            ],
            messages=[{"role": "user", "content": f"test {i}"}],
        )
        u = resp.usage
        print(
            f"req {i}: input={u.input_tokens} "
            f"create={u.cache_creation_input_tokens} "
            f"read={u.cache_read_input_tokens}"
        )
 
diagnose()

Expected output:

req 0: input=8 create=2103 read=0
req 1: input=8 create=0    read=2103
req 2: input=8 create=0    read=2103

If read becomes positive on req 1, caching is working. If it's still zero, walk through causes #1 to #5 in order — they cover almost every miss I've seen in real codebases.

For the design side of caching at scale, see my notes on cutting our monthly Claude API bill in half with prompt caching and the deeper implementation record combining Haiku 4.5, streaming, and prompt caching to cut a personal app's API costs. Pair both with API rate limit best practices when you start hitting throughput limits.

What "working" looks like once it's wired up

Once your cache is firing reliably, you'll see three patterns in your dashboards.

First, the per-request input token cost drops to roughly one-tenth of its old value for any prefix you cached, with brief spikes back to full cost every time the TTL window rolls over and a new write happens. If you're charting cost in a grid, the daily shape becomes a flat line with a few small bumps rather than the old constant ceiling.

Second, the ratio of cache_read_input_tokens to cache_creation_input_tokens tells you whether your TTL choice is right for the traffic shape. Anything above 10:1 read-to-write is a healthy steady state. If the ratio sits below 3:1, your write premium is eating most of the savings — that's usually a TTL problem (cause #3) or a prefix that drifts more than you realized (cause #4).

Third, latency improves measurably on cached prefixes — usually a 30 to 50 percent drop in time-to-first-token for the same prompt. This is the side benefit nobody talks about: prompt caching isn't only a cost feature, it's a UX feature for any product where users wait on the first chunk.

Wire those three signals into the same dashboard you already use for error rates and rate-limit headroom. Once the team can see them, regressions get caught the same week they ship rather than the next billing cycle.

A measured example: an image-metadata generation queue

The backend of an app I run as an indie developer at Dolice has a queue that generates image metadata through the Claude API. Every item ships the same ~3,200-token system prompt (classification rules and an output-format definition), and before caching, input tokens alone dominated the processing cost.

Here is the usage data aggregated over the same 1,000-image batch, before and after enabling the cache.

The detail that is easy to miss is the queue's submission interval. At first, each item paused for tens of seconds of external I/O, so the 5-minute TTL just barely survived. But in the overnight window, where the gap between items stretched close to ten minutes, cache_creation_input_tokens spiked — cause #3, exactly. Once I batched submissions into bursts and kept the interval under five minutes, the cache_read ratio stayed above 10-to-1 across every hour of the day.

The single number worth watching is whether read-to-write stays above 10-to-1. Looking back at data from a period when I assumed the cache "must be working," I found the ratio sitting around 4-to-1, with the write premium quietly eating most of the savings. Emitting the metric is what keeps you from fooling yourself.

Choosing between the two TTLs

Whether to use the 5-minute or 1-hour TTL comes down to access interval and burstiness. Here's the rule of thumb I actually use.

When in doubt, log the read-to-write ratio for 24 hours on the default 5-minute TTL first, then switch only the workloads whose ratio splits badly across the day. Flipping everything to 1 hour uniformly just loses the extra write premium on your high-frequency traffic.

A checklist for the fastest diagnostic loop

When you spot a miss, work through these in order — one at a time, watching usage after each change.

1. Count the cached prefix's tokens and confirm it clears the model's minimum (1,024 for Sonnet, 2,048 for Haiku) — cause #1
2. Confirm cache_control sits on the last block of the static prefix, with dynamic blocks after it — cause #2
3. Check request timestamps in your logs to confirm the access interval doesn't exceed the TTL (5 min by default) — cause #3
4. Print a SHA-256 hash of the prefix on every request and confirm the value is stable — cause #4
5. Confirm the tools array is emitted in a fixed order — cause #5

Running through these five once has explained nearly every cache miss I've hit in production. Causes often compound, so if one fix doesn't move the numbers, move on to the next item.

What to do next

Drop the diagnostic script into one of your live services for two minutes and watch whether cache_read_input_tokens moves. If it doesn't, start at cause #1 and work down — that's the shortest path to a fix.

Caching isn't a feature you set and forget. It's a feature you instrument with usage, then iterate on until the numbers tell you it's working. Get that loop running once and your bill will start cooperating.