Coalescing Concurrent Claude API Calls: Single-Flight Against Duplicate Inference and Cache Stampede
A design for collapsing identical prompts that fire at the same instant into a single upstream Claude call, using single-flight (request coalescing). In-process and distributed implementations, jittered retries, and negative caching, with measured results.
When Extended Thinking Flattened My Accuracy but Doubled the Bill — Field Notes on Measuring the Marginal Utility of Thinking Tokens
You pinned budget_tokens high 'to be safe,' accuracy barely moved, and the bill kept climbing. These field notes show how to ledger real thinking-token usage by p50/p95/hit-rate and measure how much accuracy each extra 1,000 thinking tokens actually buys.
A Long Non-Streaming Response Was Billed Twice Past the 10-Minute Wall: Redesigning the SDK's Default Timeout and Retries
The Anthropic SDK's default 10-minute timeout and two automatic retries can silently re-run a long non-streaming response and bill you twice. Here is how the trap works, and how to close it with streaming, explicit timeout/max_retries, and a small local ledger — with measured before/after numbers.
Your Cache Hit Rate Resets to Zero the Morning You Switch Models — Prompt Cache Rewarm Design for the Opus 4.8 to Sonnet 5 Cutover
Prompt caches are scoped per model, so day one of a model migration starts at a 0% hit rate. Why percentage-based rollouts break cache economics twice over, and how cohort cutover by task family preserves them — with working measurement code.
When Context Editing Made My Agent Re-run the Same Search — Field Notes on Clear Boundaries and Cache Invalidation
After turning on Context Editing to auto-clear tool results, the agent forgot what it had just read, re-ran the same tool, and the cache rebuilt every turn so costs went up. Field notes on instrumenting the silent regression and setting trigger, keep, and clear_at_least from measured data.
I Edited One Line of a Tool Description and the Whole Prompt Cache Rebuilt — Where to Place cache_control Breakpoints
Hit rate suddenly flatlined at zero because a volatile block sat upstream of stable ones. This walks through how prefix-cache cascade invalidation works, how to reorder blocks from stable to volatile, and where to spend your four cache_control breakpoints — with code and decision tables.
My Morning Batch Was Missing the Prompt Cache Every Time — Warming Cadence and the Break-Even Math for the 1-Hour TTL
Jobs that run a few hours apart cold-miss the prompt cache even with a 1-hour TTL. Here is how to back out the right warming interval from the TTL, and how to write the break-even formula that decides whether warming pays off — with numbers from a four-site daily generation pipeline.
When Claude API Prompt Caching Quietly Stops Hitting in Production — Field Notes on TTL and Measured Savings
Prompt caching works beautifully the day you ship it, then quietly stops hitting in production. The five things that break the prefix, how to choose between 5-minute and 1-hour TTL, and how to measure real savings from usage instead of guessing.
Don't Carry Search Results Twice: Trimming Consumed Blocks with response_inclusion
When an agent runs dynamic filtering, output tokens balloon because the raw search-result blocks a code execution call already consumed get echoed back into the response. Here is when response_inclusion: excluded is safe to use, when you must keep full, with implementation and a decision table.
Retiring the 'Please Continue' Prompt — Single-Pass Long-Form Generation with Claude Fable 5's 128k Output
Every month my report generation hit the output cap and I had to ask Claude to 'continue from here.' Claude Fable 5's 128k output let me retire that workflow: a streaming implementation, a resume-after-disconnect pattern, and a measured cost comparison against chunked generation.
Splitting Claude API prompt cache into 5m and 1h tiers — separate TTLs cut cost and stabilize ops
Anthropic's cache_control supports two TTLs: 5 minutes and 1 hour. Splitting them into a two-tier layout — 1h for static system/tools, 5m for variable few-shot — meaningfully changed both my costs and my on-call life. Here's the design with the numbers I observed.
A Two-Tier Setup — Haiku 4.5 Orchestrator with Opus 4.6 Worker for Balancing Cost and Quality
How an indie developer's two-tier setup — Haiku 4.5 as the orchestrator and Opus 4.6 as the worker — cuts monthly API spend by roughly 70% without sacrificing the quality readers pay for.