Claude API 429 Errors in Production: Lessons from Six Parallel Content Pipelines

Four Pitfalls the Docs Don't Mention

Four things that the official documentation does not call out clearly tripped me up during implementation.

1. Retry-After tends to be shorter than reality

Empirically, if you sleep exactly Retry-After seconds and retry, you re-hit the same 429 about 20–30% of the time. The server-side limiting behaves more like a rolling window than a fixed minute bucket, so landing exactly on the boundary often gets rejected by a small calculation drift. The fix is simple: always add positive jitter. I use Retry-After * (1.2 + random(0, 0.3)).

2. Some 429s arrive without Retry-After

When 429 coincides with overload (HTTP 529 territory), the Retry-After header may be missing. If your code calls float() on the header without a null check, it will throw. Always wrap it with a default — I use 30 seconds.

3. Parallel retries synchronize and burst

With four concurrent in-flight requests, if all four hit 429 together, their retry timers fire together too, and you re-launch four parallel calls the moment recovery hits. That cascades and 429s avalanche. Independent random jitter per parallel call eliminates this entirely.

4. RateLimit-Reset-* and Retry-After carry different precision

Retry-After is an integer of seconds, rounded. RateLimit-Reset-* is an absolute ISO8601 timestamp and often retains millisecond precision. For long polling loops, trust the Reset-* side; it converges faster.

Python Retry: Exponential Backoff with Jitter and Idempotency Key

This is the minimal version of the retry implementation actually running in my pipelines. It pairs with the official Anthropic SDK.

import time
import random
import hashlib
import json
import anthropic
from anthropic import RateLimitError, APIStatusError
 
client = anthropic.Anthropic()
 
def idempotency_key(model: str, messages: list, max_tokens: int) -> str:
    """Same request -> same key. Dedup is the caller's job."""
    payload = json.dumps({"m": model, "ms": messages, "t": max_tokens}, sort_keys=True)
    return hashlib.sha256(payload.encode()).hexdigest()[:24]
 
def call_with_retry(model, messages, max_tokens=1024, max_retries=5):
    """Jittered exponential backoff with Retry-After honoring."""
    base = 1.0  # base seconds
    key = idempotency_key(model, messages, max_tokens)
 
    for attempt in range(max_retries):
        try:
            resp = client.messages.create(
                model=model,
                max_tokens=max_tokens,
                messages=messages,
                extra_headers={"X-Idempotency-Key": key},
            )
            return resp
        except RateLimitError as e:
            retry_after = e.response.headers.get("Retry-After") if e.response else None
            if retry_after:
                wait = float(retry_after)
            else:
                wait = base * (2 ** attempt)
 
            # 1. positive jitter (avoids window-edge collisions)
            wait = wait * (1.2 + random.random() * 0.3)
            # 2. cap (prevents runaway 60s+ sleeps)
            wait = min(wait, 60.0)
 
            if attempt == max_retries - 1:
                raise
            time.sleep(wait)
        except APIStatusError as e:
            # 529 overloaded -- same treatment as 429
            if e.status_code != 529:
                raise
            time.sleep(base * (2 ** attempt) * (1.2 + random.random() * 0.3))
 
    raise RuntimeError("retry exhausted")

Three things to hold steady. Jitter must be positive (at least 1.2x); cap the sleep at around 60 seconds; attach X-Idempotency-Key to each request so your own logs can dedupe in case the same call fires twice. Anthropic does not officially support an idempotency key — this is for your own ledger.

TypeScript Version (Runs on Cloudflare Workers)

The back end of my sites runs on Cloudflare Workers, so I keep a TypeScript port. It uses @anthropic-ai/sdk directly.

import Anthropic from "@anthropic-ai/sdk";
 
const client = new Anthropic();
 
async function sleep(ms: number) {
  return new Promise((r) => setTimeout(r, ms));
}
 
export async function callWithRetry(
  model: string,
  messages: Anthropic.MessageParam[],
  maxTokens = 1024,
  maxRetries = 5
) {
  const base = 1000; // ms
 
  for (let attempt = 0; attempt < maxRetries; attempt++) {
    try {
      return await client.messages.create({
        model,
        max_tokens: maxTokens,
        messages,
      });
    } catch (err) {
      const status = (err as any)?.status;
      const retryAfter = (err as any)?.headers?.["retry-after"];
 
      if (status !== 429 && status !== 529) throw err;
      if (attempt === maxRetries - 1) throw err;
 
      let waitMs = retryAfter
        ? parseFloat(retryAfter) * 1000
        : base * 2 ** attempt;
      // positive jitter + cap
      waitMs = Math.min(waitMs * (1.2 + Math.random() * 0.3), 60_000);
 
      await sleep(waitMs);
    }
  }
  throw new Error("retry exhausted");
}

setTimeout precision is modest under Workers, which is another reason to keep jitter positive — it ensures you clear window edges.

Token Bucket: Throttle Yourself Before the Server Does

Retrying after a 429 is a reactive design. Under heavy traffic, it pays to throttle yourself first. In my pipeline each site has its own token bucket, and the refill rate drops automatically when RateLimit-Remaining-Tokens falls below a threshold.

import time
import threading
 
class TokenBucket:
    """Self-imposed rate limit. Survives bursts; smooths long-running batches."""
    def __init__(self, capacity: int, refill_per_sec: float):
        self.capacity = capacity
        self.tokens = float(capacity)
        self.refill = refill_per_sec
        self.last = time.monotonic()
        self.lock = threading.Lock()
 
    def take(self, cost: int = 1, timeout: float = 30.0) -> bool:
        deadline = time.monotonic() + timeout
        while True:
            with self.lock:
                now = time.monotonic()
                delta = now - self.last
                self.tokens = min(self.capacity, self.tokens + delta * self.refill)
                self.last = now
                if self.tokens >= cost:
                    self.tokens -= cost
                    return True
                short = cost - self.tokens
                wait = short / self.refill
            if time.monotonic() + wait > deadline:
                return False
            time.sleep(min(wait, 1.0))
 
    def slow_down(self, factor: float = 0.5):
        """Called when RateLimit-Remaining-Tokens drops below threshold."""
        with self.lock:
            self.refill *= factor
 
# usage
tpm_bucket = TokenBucket(capacity=90_000, refill_per_sec=90_000 / 60)
 
def call(model, messages, max_tokens=1024):
    estimated = sum(len(m["content"]) for m in messages) // 3 + max_tokens
    if not tpm_bucket.take(estimated, timeout=60):
        raise RuntimeError("local bucket starved -- backpressure upstream")
    resp = call_with_retry(model, messages, max_tokens)
    remaining = int(resp.response.headers.get("RateLimit-Remaining-Tokens", 0))
    if remaining < 10_000:
        tpm_bucket.slow_down(0.6)
    return resp

Three points to hold:

1. The token estimate "characters / 3 + max_tokens" is intentionally crude. It is safer to over-count than under-count, especially with mixed-language prompts.
2. The refill rate is set to 90% of the TPM ceiling. Setting it to 100% guarantees boundary rejections.
3. Drop the refill rate dynamically when RateLimit-Remaining-Tokens falls below threshold. This prevents the "I rate-limit myself by using up my own budget" failure mode.

When to Move Jobs to the Batch API

For jobs that don't need real-time responses, I recommend moving them to the Batch API (Messages Batches). The criteria I apply:

• Response not needed within 60 seconds: use Batch. The effective rate budget is roughly 50% looser and cost is roughly half. I moved post-publication integrity checks and SEO reference generation here.
• Real-time needed, but 100+ requests in a chain: use the standard API with a token bucket. Batch's completion window (up to 24 hours) is unacceptable here.
• Real-time needed, 10 or fewer one-off requests: standard API with retry. Batch overhead isn't worth it.

On my own six-site setup, after migrating eligible jobs to Batch, the 429 rate dropped from about 0.42% to 0.04%. Batch doesn't fail the whole job on error — each request returns succeeded or errored independently — so retry design is actually simpler than with the standard API.

Cut the Token-Side 429s — Offload Reusable Context to Prompt Caching

Of the three axes, retry design and the token bucket help with requests-per-minute and concurrency, but tokens-per-minute (TPM) only comes down if you shrink what you actually send. If a long system prompt or a shared document goes out in full on every call, you can throttle request count and still hit 429 on the TPM axis.

This is where prompt caching earns its place. Send the large, reusable context with cache_control, and from the second call onward it counts as a cache read, lowering input tokens for both billing and TPM.

import anthropic
 
client = anthropic.Anthropic()
 
# A large shared preface reused across many calls (thousands of tokens)
SHARED_CONTEXT = """(product docs, a style guide, the same preamble every time)"""
 
def ask_with_cache(question: str) -> anthropic.types.Message:
    resp = client.messages.create(
        model="claude-sonnet-4-6",
        max_tokens=1024,
        system=[
            {"type": "text", "text": "You are a careful assistant."},
            {
                "type": "text",
                "text": SHARED_CONTEXT,
                "cache_control": {"type": "ephemeral"},  # cache this block
            },
        ],
        messages=[{"role": "user", "content": question}],
    )
    u = resp.usage
    # A rising cache_read_input_tokens means less TPM pressure
    print(u.input_tokens, u.cache_read_input_tokens, u.cache_creation_input_tokens)
    return resp

The catch is that the cache only hits when the prefix matches the previous call exactly. If the preface differs per site, the cache is rebuilt every time: cache_creation_input_tokens climbs while reads stay flat. As an indie developer running all six sites myself under Dolice, once I pinned the shared instructions into a single fixed template, the TPM-driven 429s dropped noticeably. Shape how you send with retries, the token bucket, and the Batch API first, then trim how much you send with caching — that order tends to settle cleanly in practice.

Production Metrics and Pre-Deploy Checklist

These are the metrics I always measure and the checks I run before adding a new call site.

Metrics (aggregated daily)

1. 429 rate (by time-of-day and by model)
2. Recovery rate after retry (1st attempt, within 3, within 5)
3. Retry-After median and P95
4. Minimum RateLimit-Remaining-Tokens seen
5. Token-bucket starvation count

Before adding a new call site

1. Does the retry use positive jitter? Pure exponential backoff alone won't cut it.
2. Is there a default when Retry-After is null?
3. Is the retry capped (I use 5 attempts plus a 60-second per-attempt cap)?
4. Is the token estimate being fed into the bucket?
5. Could this job be moved to Batch API? Ask once, every time.

Closing Notes for Production

If I were to highlight one thing from operating six pipelines: do not trust Retry-After alone. The server-side hint is correct, but it tends to fall short on the rolling-window boundary, and you must absorb that with positive jitter. With a self-imposed token bucket in front, and Batch API behind eligible jobs, 429s become rare enough to ignore in day-to-day operations.

My own six-site setup is still being tuned, but once 429 stops gating your scheduled tasks, the psychological load of running content automation drops considerably. I hope this is useful to anyone running indie pipelines at a similar scale. Thank you for reading.