The 200K-Token Cliff That Doubled My Nightly Bill — A count_tokens Preflight to Avoid Long-Context Pricing

The preflight in Python

Here's the function I call right before sending. It estimates input tokens and returns one of three decisions, including a safety margin.

import os
import anthropic
 
client = anthropic.Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])
 
TIER_BOUNDARY = 200_000   # where long-context pricing kicks in
SAFE_CEILING = 180_000    # safety margin; I run 10% below the edge
MODEL = "claude-sonnet-5"
 
def count_input_tokens(system, messages, tools=None):
    """Estimate input tokens before sending.
    count_tokens is not billed and does not consume the rate limit."""
    resp = client.messages.count_tokens(
        model=MODEL,
        system=system,
        messages=messages,
        tools=tools or [],
    )
    return resp.input_tokens
 
def preflight(system, messages, tools=None):
    tokens = count_input_tokens(system, messages, tools)
    if tokens <= SAFE_CEILING:
        return "send", tokens    # send as-is
    if tokens <= TIER_BOUNDARY:
        return "warn", tokens    # right at the edge; log and send
    return "block", tokens       # into long-context pricing; needs handling
 
decision, tokens = preflight(system_prompt, history)
print(f"input={tokens:,} decision={decision}")
 
# Example output:
#   input=176,320 decision=send
#   input=204,880 decision=block

I cap the practical limit at 180K rather than exactly 200K for a reason. The count_tokens estimate and the tokens you're actually billed for don't match perfectly. Tool definitions, the system prompt, and prompt-cache handling introduce a drift of a few thousand tokens. If you hug the edge, an estimate under 200K can still cross on the real send. This 10% margin insures against that uncertainty.

What to do once you've stopped at the edge

When block fires, you have three main options. I choose based on the nature of the input.

1. Compact by summarizing, back inside the boundary. Effective when conversation history has piled up. Fold the older turns into a single summary with a cheaper model, and keep only the most recent turns verbatim.

def compact_history(messages, keep_recent=6):
    """Fold older turns into a summary to get back inside the boundary.
    The most recent keep_recent turns stay verbatim."""
    if len(messages) <= keep_recent:
        return messages
    old, recent = messages[:-keep_recent], messages[-keep_recent:]
    rendered = "\n".join(f'{m["role"]}: {m["content"]}' for m in old)
    summary = client.messages.create(
        model="claude-haiku-4-5",   # a cheap model is plenty for summaries
        max_tokens=1024,
        messages=[{
            "role": "user",
            "content": "Summarize this conversation as bullet points, keeping only facts needed downstream:\n" + rendered,
        }],
    ).content[0].text
    return [{"role": "user", "content": f"[Summary so far]\n{summary}"}] + recent

2. Split across multiple requests. If you'd been cramming independently processable files into one request, break them into chunks under 200K each. Several in-boundary requests cost less in total than one giant request that straddles the boundary, because you never trip the whole-request repricing at all.

3. Defer the run. If the batch doesn't strictly need to finish that night, revisiting the split strategy by hand the next day can be more honest than degrading quality with over-aggressive compaction. Forcing everything inside the boundary and summarizing away the context you actually need defeats the purpose.

Whichever you pick, I've come to treat crossing the boundary not as an anomaly but as a normal branch to design for.

The TypeScript version

For pipelines I run on Node, the same decision in TypeScript.

import Anthropic from "@anthropic-ai/sdk";
 
const client = new Anthropic();
const TIER_BOUNDARY = 200_000;
const SAFE_CEILING = 180_000;
const MODEL = "claude-sonnet-5";
 
type Decision = "send" | "warn" | "block";
 
async function preflight(
  system: string,
  messages: Anthropic.MessageParam[],
): Promise<{ decision: Decision; inputTokens: number }> {
  const { input_tokens } = await client.messages.countTokens({
    model: MODEL,
    system,
    messages,
  });
  let decision: Decision = "send";
  if (input_tokens > TIER_BOUNDARY) decision = "block";
  else if (input_tokens > SAFE_CEILING) decision = "warn";
  return { decision, inputTokens: input_tokens };
}
 
// Usage
const { decision, inputTokens } = await preflight(systemPrompt, history);
console.log(`input=${inputTokens.toLocaleString()} decision=${decision}`);
if (decision === "block") {
  throw new Error("Crossed the long-context boundary; compact or split first");
}

I've found this preflight sits well upstream of a budget circuit breaker that hard-stops on daily token overruns. If the circuit breaker is the last line of defense that stops you once you've overspent, the preflight is the steering that keeps you off the pricing cliff in the first place.

Pitfalls from real operation

A few things I noticed after running this for a couple of weeks.

The gap between estimate and actual billing comes mainly from tool definitions and prompt caching. The more tools a batch passes, the more the real send tends to exceed the count_tokens figure by a few thousand tokens. Cache-hit tokens are counted differently as input, so when caching is on you need to keep "the raw token count for boundary checks" separate from "the actual amount billed." I've written up how to make caching pay off in Halving monthly cost with prompt caching.

One more thing: watch the distribution over days, not the absolute input count. A batch that sits steadily around 120K but spikes to 190K one night will cross the boundary the following week. I log every night's preflight result and, when the frequency of warn starts climbing, revisit the compaction logic. The cliff doesn't appear suddenly — it creeps closer.

For deciding when to use 1M context from the Claude Code side, I've covered that separately in When to use native 1M context, decided by cost. Hitting the API directly and going through Claude Code call for slightly different cost instincts.

The next step

Start by adding a single count_tokens call right before the send in just one of your live unattended batches, and log the result. You don't need to stop anything yet. Just watching the token distribution over a few days shows you how close your pipeline actually runs to the cliff.

Until I fitted this instrument myself, I'd walked past the edge many times without noticing. Simply making it visible let me hand the nightly runs off with far more peace of mind.