Coalescing Concurrent Claude API Calls: Single-Flight Against Duplicate Inference and Cache Stampede
A design for collapsing identical prompts that fire at the same instant into a single upstream Claude call, using single-flight (request coalescing). In-process and distributed implementations, jittered retries, and negative caching, with measured results.
Designing Around Claude API 413 request too large — Preflight Sizing and Splitting
Pack too much text, images, and tool_result into one request and Claude API rejects it with 413 request too large. Here is a code-backed design for measuring request bytes before you send, telling the two kinds of 413 apart, and splitting requests without breaking them.
Is the Draft That Passed the Gate the Same One You Published?
In unattended pipelines, the file your quality gate inspected and the file you actually publish can quietly diverge. Here is a digest-bound gate receipt design, with working code and measured results from 180 days of running it.
When Claude API Suddenly Starts Returning 429 in Production — Field Notes on Measuring Rate-Limit Headroom from Headers and Throttling Before You Run Dry
Scrambling to add retries after a 429 is always a step behind. Claude API writes how much you have left into every response header. These are field notes on measuring that headroom continuously and throttling yourself before it runs out.
How Many Concurrent Claude API Requests Can You Actually Hold? Sizing Production Infrastructure with Little's Law and Measured Memory
Concurrency, queue depth, and memory are numbers you can derive, not guess. A working method for sizing Claude API production deployments with Little's Law, a memory probe, and a 30-minute load check — learned the hard way from an OOM crash.
Five Minutes of Silence, and Something Retries on Your Behalf — Rethinking Retry Ownership After the Streaming Idle Watchdog Became a Default
Claude Code's streaming idle watchdog is now on by default, quietly adding another retrying layer to your stack. This article inventories the four layers (SDK, wrapper, watchdog, scheduler), computes worst-case attempt amplification, and shows how to collapse retry ownership into a single layer.
When Claude API Document Extraction Is Confidently Wrong — Field Notes on Catching Silent Errors with Invariants
In structured extraction from invoices and contracts, the real danger isn't a crash — it's a value that's silently wrong while the schema validates and confidence reads high. Field notes on invariants, two-pass extraction, and tracking field-level error rates.
A Silent Drop to a Weaker Model Is Scarier Than an Error: Designing a Capability Floor for Claude API Fallback
When a model becomes unavailable in an unattended pipeline, automatically dropping to a weaker model is dangerous. Drawing on years of running automated indie pipelines, this is how to use per-task capability contracts and a degradation budget to decide where to stop.
Designing the Give-Up Condition in Self-Repair Loops: Four Error Classes, Four Retry Budgets
LLM self-repair loops break on the fantasy that 'if you keep fixing, it eventually passes.' Classify errors into four classes, give each its own retry budget. Working TypeScript and real cost numbers included.
When Claude API Streaming Stops Without an Error: Detecting Silent Stalls and Resuming Mid-Stream
How to catch the 'silent stall' where Claude API streaming stops with no exception at all, using a content-level watchdog that times the gap between tokens, plus a resume path that carries received text forward as an assistant prefill, and a four-layer timeout budget for long-running automation.
When the Previous Run Hasn't Finished and the Next One Starts: Leases and Fencing Tokens for Scheduled Agents
A scheduled agent that runs on a fixed clock can overtake itself and start twice. From the moment a naive lock breaks to leases, fencing tokens, and bounded catch-up — worked through with the implementation I actually run.
When Your Support AI Is Confidently Wrong in Production — Notes on Refusing Outside Its Grounding and Routing to Humans
A field-tested approach to the Claude API support agent that demos perfectly yet states non-existent facts in production. Covers deciding 'don't answer' at retrieval time, grounded generation, measuring confident-wrong rate, and tuning escalation precision.