All Articles
Don't Trust the Confidence Score: Per-Class Calibration and Abstain Routing for Vision Classification
Overall accuracy looked fine while individual categories quietly collapsed. Here is how I calibrated Claude Vision's self-reported confidence per class and routed abstentions to a human queue.
My Nightly Batch Was Quietly Running on a Bigger Model — Declaring Per-Task Model Ceilings and Enforcing Them
Scatter model names through your code and a cheap batch job will eventually run on an expensive model. Here is a manifest that declares a ceiling per task, a deny-by-default resolver, and a morning audit that catches drift — with working code.
Same Output, Different Path — Guarding Agent Trajectories with Invariants
When the default model changes, your final output can stay correct while the path your agent takes quietly shifts. Here is a trajectory regression harness built on recorded tool traces and deterministic invariants, with working code and measured numbers.
When the RAG Started Being Confidently Wrong — Field Notes on Measuring Retrieval Misses With Groundedness
In a Claude API RAG, the answers stay fluent while the facts drift. Often the cause is a silent recall decay on the retrieval side, missing the document that holds the answer. Field notes on measuring groundedness and retrieval hit rate and walking the system back, with working code and real numbers.
The 200K-Token Cliff That Doubled My Nightly Bill — A count_tokens Preflight to Avoid Long-Context Pricing
Sonnet 5's native 1M context switches the entire request to long-context pricing the moment input crosses 200K tokens. Here's how I caught that silent cost cliff before sending, using the billing-exempt count_tokens API, with working Python and TypeScript code.
Rolling Out Agent Behavior Changes Gradually with Feature Flags
When an unattended agent changes its prompt or behavior all at once, quiet regressions hide until morning. Here is a design for assigning new behavior to only a fraction of runs, keeping a control group, and adding deterministic bucketing, canary comparison, and automatic rollback.
Contract-Test Every Tool Before You Submit or Automate an MCP Connector
A connector that works once in a chat can still break silently in an unattended job through misread response shapes or double-fired writes. Here is a small harness that machine-checks tool descriptions, response contracts, idempotency, and latency, with measured numbers.
Four sites, one Claude bill: attributing spend per workspace to decide which pipeline to trim
When several projects share one Claude organization, the bill arrives as a single number that hides which pipeline is expensive. Field notes on splitting that spend per workspace with the Cost and Usage Report group_by, defensively parsing the results, and deciding what to trim by cost-effectiveness.
When Claude API Suddenly Starts Returning 429 in Production — Field Notes on Measuring Rate-Limit Headroom from Headers and Throttling Before You Run Dry
Scrambling to add retries after a 429 is always a step behind. Claude API writes how much you have left into every response header. These are field notes on measuring that headroom continuously and throttling yourself before it runs out.
Fable 5 Is Back Worldwide and Sonnet 5 Is the Default — Where Each of the Three Models Belongs in a Solo Automation Stack
With Fable 5 redeployed worldwide and Sonnet 5 now the default, solo automation suddenly has three capable top-tier models to reach for. Instead of ranking them, this piece assigns each a role and captures that in a policy object with a fallback ladder and run-level logging.
Don't Let the Opus 4.7 Fast Mode Retirement (July 24) Kill Your Unattended Jobs
claude-opus-4-7 fast mode retires on 2026-07-24, and speed: fast starts throwing errors. Here's how to keep unattended pipelines from breaking silently: mechanically detect where fast mode is used, add a fail-closed runtime guard, and migrate to 4.8 with working code.
Reading the Claude apps gateway Announcement, I Rebuilt My Indie-Scale Control Plane
The self-hosted Claude apps gateway is a control-plane/data-plane separation you can scale down. Per-app cost attribution, model allowlists, and fail-closed spend caps, implemented as a small Cloudflare Workers proxy.