How I Cut My Claude API Bill in Half With Prompt Caching

I have been running prompt caching in production for about six months now. I was skeptical at first, but on workloads where every request carries a long system prompt, my bill genuinely landed at less than half of what it used to be. The catch is that the wrong design produces almost no savings at all — it is one of those features with a sharp gap between "configured properly" and "configured anyway."

This article is a write-up of what I learned introducing prompt caching across several of my own apps, watching the monthly bill move, and rewriting the prompt structure until it actually paid off. I have included the behaviors I picked up from real traffic that the official docs do not really spell out.

Why I started taking prompt caching seriously

The honest reason: my API bill came in at roughly three times what I expected. Chat-style services tend to grow very long system prompts — context, persona, output rules — and mine had drifted to about 6,000 to 10,000 tokens. I was sending the whole thing on every turn.

Twenty turns of conversation means I was paying for north of 120,000 tokens of system prompt alone. It was hidden in the noise of context handling on each request, but once I broke down the bill it was painful to look at.

Prompt caching attacks exactly this "send the same preamble every time" cost. A cache hit charges input tokens at one tenth of the standard rate (cache writes cost 1.25x, but only on first write).

That said, whether you actually get savings depends heavily on where the static portions of your prompt live and how your traffic accesses the cache.

What kind of prompt structure actually caches

Anthropic's cache_control parameter lets you mark a portion of the prompt as "cache up to here." In the Python SDK it looks like this.

from anthropic import Anthropic
 
client = Anthropic()
 
response = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    system=[
        {
            "type": "text",
            "text": LONG_STATIC_SYSTEM_PROMPT,  # several thousand tokens
            "cache_control": {"type": "ephemeral"}
        }
    ],
    messages=[
        {"role": "user", "content": user_input}
    ]
)

Everything up to and including the marked element gets cached. The crucial point: a cache only hits when the prompt matches exactly from the very beginning up to the marked position. If anything user-specific lives before the cache marker, you will miss every time.

This was my first stumble. I had been embedding usernames and session metadata at the top of my system prompt. The cache existed; I just never hit it.

The right ordering is: static content first, dynamic content after.

system=[
    {
        "type": "text",
        "text": STATIC_PROMPT,        # persona, rules, examples — never changes
        "cache_control": {"type": "ephemeral"}
    },
    {
        "type": "text",
        "text": f"User context: {user_id}, time: {now}"  # dynamic
    }
]

Start by measuring hit rate

The very first thing to do after wiring caching up is measure your hit rate. The response usage object contains the fields you need.

print(response.usage)
# Usage(
#   input_tokens=120,
#   cache_creation_input_tokens=0,
#   cache_read_input_tokens=8500,
#   output_tokens=300
# )

cache_read_input_tokens is what hit the cache. cache_creation_input_tokens is what got freshly written. input_tokens is regular billed input.

I log at minimum these metrics on every request.

def log_cache_metrics(response, request_id):
    usage = response.usage
    total_input = (
        usage.input_tokens
        + usage.cache_creation_input_tokens
        + usage.cache_read_input_tokens
    )
    hit_rate = usage.cache_read_input_tokens / total_input if total_input else 0
 
    logger.info({
        "request_id": request_id,
        "cache_hit_rate": hit_rate,
        "cache_read": usage.cache_read_input_tokens,
        "cache_write": usage.cache_creation_input_tokens,
        "input_uncached": usage.input_tokens,
        "output": usage.output_tokens,
    })

If the rolling hit rate stays above 80 percent your design is in roughly the right shape. If it drops under 50 percent, almost always something dynamic has crept in front of the cache marker — go re-read the prompt assembly code.

Picking a TTL

The default TTL for an ephemeral cache entry is five minutes since last access. For continuous chat that is fine, but consider an internal tool that only sees traffic during weekday lunch breaks — every break, the cache evaporates and gets rewritten.

For longer TTL (one hour) you pass ttl inside cache_control.

"cache_control": {"type": "ephemeral", "ttl": "1h"}

The catch: writes against a one-hour TTL cost twice as much as the standard write. If your hit rate is high it is a clear win; if your hit rate is low, you are now paying more on writes too.

My rule of thumb: if a typical user touches the system at least ten times a day, the one-hour TTL pays for itself. For overnight batch jobs that have no follow-up traffic, the default five minutes is right. Read your access logs before flipping the switch.

Mind the token floor

There is a minimum cacheable size: 1,024 tokens for the Sonnet family, 2,048 for Haiku. Below that, cache_control does nothing.

A subtle trap is dynamically assembled system prompts. If you grow or shrink rules based on user role, you can land at 2,000 tokens one day and 900 tokens the next, and the second day silently bills you full price.

In production I padded my system prompt template so that it always lands above 1,024 tokens. I did this honestly — by enriching the examples section — and pushed all per-user content past the cache boundary. Hit rate became stable.

The four-block ceiling

You can mark at most four blocks per request with cache_control. If you carve up a long RAG document carelessly, you will hit the ceiling without realizing it.

The pattern that has worked best for me: split along logical boundaries, four or fewer blocks total.

system=[
    {"type": "text", "text": COMPANY_RULES, "cache_control": {"type": "ephemeral"}},
    {"type": "text", "text": OUTPUT_FORMAT_SPEC, "cache_control": {"type": "ephemeral"}},
    {"type": "text", "text": EXAMPLES, "cache_control": {"type": "ephemeral"}},
    {"type": "text", "text": USER_RAG_CONTEXT, "cache_control": {"type": "ephemeral"}},
]

Each block forms an independent "cache layer." If only USER_RAG_CONTEXT changes, the three blocks before it still hit. This is enormously useful for bots that hold long-lived background knowledge.

The day I added one tool and hit rate went to zero

This was the most expensive lesson of the six months. I shipped one small tool to the support bot. I did not touch a single character of the prompt. The next morning, hit rate was zero.

Cache matching walks the prefix in a fixed order: tools, then system, then messages. Tool definitions sit at the very front, so changing them invalidates everything behind them — including the system block you carefully marked.

client.messages.create(
    model="claude-sonnet-4-6",
    tools=[search_tool, calc_tool],   # one addition here
    system=[
        {"type": "text", "text": STATIC_PROMPT,
         "cache_control": {"type": "ephemeral"}},   # rewrites this too
    ],
    messages=[...],
)

My fix was procedural rather than clever: tool changes ship on a weekly release, and that day is expected to run cold. The monitoring threshold is relaxed on release days so the alert does not cry wolf. Working as an indie developer, there is no release train to hide behind, so the rule had to be one I imposed on myself.

The flip side is worth noting. If your tool definitions are stable, put a cache_control marker at the end of tools and cache the definitions themselves. Ten tools with verbose JSON Schemas add up to thousands of tokens on their own.

Cache the conversation history too

While I was only caching the system prompt, I was leaving about half the savings on the table. In a twenty-turn conversation the messages array keeps growing. By turn ten the history alone runs into thousands of tokens — and every one of them was being recomputed on every call.

cache_control works on message blocks as well. Mark the most recent assistant reply and everything up to that point gets cached.

def with_history_cache(messages: list[dict]) -> list[dict]:
    msgs = [dict(m) for m in messages]
    if not msgs:
        return msgs
    last = msgs[-1]
    content = last["content"]
    if isinstance(content, str):
        content = [{"type": "text", "text": content}]
    content = [dict(c) for c in content]
    content[-1]["cache_control"] = {"type": "ephemeral"}
    last["content"] = content
    return msgs

On the next turn, everything before that marker — system prompt and prior history alike — reads from cache. You only pay full price for the newest exchange.

The thing to watch is the four-block ceiling. I pin mine at three: one on the system prompt, one on the FAQ corpus, one at the tail of the history, with the fourth left as headroom. The history marker moves to the new tail each turn rather than accumulating.

Since making that change, per-turn input cost in long conversations flattened out. It used to climb the longer a conversation ran. Now it is close to level.

A working RAG pattern

Here is the layout I actually use in a small support bot of mine — a basic RAG that loads a FAQ corpus and answers from it.

def build_messages(user_query: str, retrieved_docs: list[str]) -> dict:
    return {
        "system": [
            {
                "type": "text",
                "text": SYSTEM_PROMPT_BASE,  # persona + output rules
                "cache_control": {"type": "ephemeral"}
            },
            {
                "type": "text",
                "text": FAQ_KNOWLEDGE_BASE,  # ~50KB fixed FAQ
                "cache_control": {"type": "ephemeral"}
            },
            {
                "type": "text",
                "text": "\n".join(retrieved_docs)  # changes per query
            }
        ],
        "messages": [
            {"role": "user", "content": user_query}
        ]
    }

With this layout the FAQ block holds a hit rate above 95 percent. The retrieved docs are uncached by definition, but that is unavoidable — they do change per query.

In aggregate, my monthly cost dropped to 38 percent of the pre-caching baseline. Closer to a third than a half.

Five things to check when caching is not hitting

This is the checklist I have written for myself more than once.

First: is anything dynamic appearing before your cache_control marker? This is where I tripped initially.

Second: are you above the minimum token threshold? Sonnet will not cache below 1,024 tokens.

Third: has the TTL expired? Five minutes of idle time and the entry is gone.

Fourth: is the model the same? claude-sonnet-4-6 and claude-sonnet-4-5 keep separate caches, so a model bump effectively resets you.

Fifth: is the prompt byte-identical? A single different whitespace breaks the match. If your prompt is templated, suspect the template function.

A real cost trajectory

Numbers from my support bot (about 12,000 requests/month) for context:

Roughly 62 percent off the baseline. The row worth staring at is the one-week mark: caching was switched on and working, but a 30 percent hit rate only bought a 24 percent reduction. Same feature, same code path — the boundary placement alone moved the outcome by a factor of three.

The longer your prompt, the bigger the payoff, so services leaning on long system prompts should look here first.

Break-even arrives sooner than you think

"Writes cost 1.25x" makes it sound like you need dozens of reads before the math works. I believed that too, and skipped caching on low-traffic endpoints because of it. Then I actually did the arithmetic and found the opposite.

Call standard input 1.0. A write is 1.25 and a read is 0.1. So a write costs an extra 0.25 — and every hit saves 0.9.

A single hit pays back a five-minute write. Two hits pay back an hour-long one.

Which means the real risk was never the write premium — it is writing a cache that expires without ever being read. Zero hits leaves you 0.25x down (1.0x on the hour TTL) and nothing more. The question to ask is not "how many times will this be read?" but "will it be read at all?" Once I reframed it that way, every endpoint I had been hesitating over got caching.

Working backward from hit rate to the bill

The "one week in" row of the table kept nagging at me, so I wrote out the relationship between hit rate and input cost as plain arithmetic. The assumptions are simple: for the cacheable part, a hit costs a read (0.1x), and a miss costs a write (1.25x on the 5-minute TTL, 2x on the 1-hour TTL). Output tokens and the uncached dynamic part are left out.

def input_cost_ratio(hit_rate: float, write_mult: float) -> float:
    """Cost of the cacheable part as a multiple of the uncached cost (1.0).
    A hit costs 0.1 (read); a miss costs write_mult (write)."""
    return hit_rate * 0.1 + (1 - hit_rate) * write_mult
 
 
def break_even(write_mult: float) -> float:
    """Hit rate at which the multiple drops below 1.0."""
    return (write_mult - 1.0) / (write_mult - 0.1)
 
 
for p in (0.3, 0.5, 0.8, 0.92, 0.95):
    r5 = input_cost_ratio(p, 1.25)
    r1 = input_cost_ratio(p, 2.0)
    print(f"{p:>4.0%}  5 min: {r5:.3f}  1 hour: {r1:.3f}")

Here is the output from my machine, expressed as a multiple of the uncached cost (1.0).

The break-even hit rate is about 22 percent on the 5-minute TTL and about 53 percent on the 1-hour TTL. This is the same fact as the "two hits pay back a 1-hour write" table above, seen from another side.

Two things stood out.

The first is the 30 percent row. The cacheable part of the input barely drops by a tenth, so a 24 percent cut on the invoice was actually a respectable result. When you feel that you added the feature and nothing got cheaper, the design may not be wrong — your hit rate may simply be sitting just past the break-even line.

The second is when the 1-hour TTL is worth choosing. Around a 50 percent hit rate, the 1-hour TTL costs more than the 5-minute one. My support bot climbed to 92 percent after I lengthened the TTL, but most of that gain probably came from moving the boundary, not from the TTL itself. A longer TTL is a trade: you pay a higher write price in exchange for a higher hit rate, so you cannot judge it without putting the before and after hit rates side by side.

My current order of operations is to measure hit rate on the 5-minute TTL first, and switch to 1 hour only when I can see the rate clearly clearing the 53 percent line.

Where to start

If reading this you suspect your service might benefit, pull a day of request logs and look at two things: the size of your system prompt, and how often it repeats. If you are above 1,024 tokens and shipping the same context repeatedly, the win is almost guaranteed.

The implementation is essentially "make system a list and add cache_control," which you can finish in an afternoon. The hard part is the design — where to draw the boundary, and how to measure. Keep this checklist nearby and walk the boundary back and forth while watching hit rate.

If you can land at 80 percent hit rate within the first few days, the cost reduction is not far behind.