When a Model Disappears Without Warning: A State Machine for Retirement, Withdrawal, and Overload

Define the State Machine Transitions

Next, make explicit how each model's availability transitions. Leave it vague and you end up deciding "when do we revert?" by gut feel during operations, losing reproducibility.

I defined the transitions as follows.

1. available → overloaded: on a run of consecutive 429 or 529. Record a timestamp; mark as a candidate to auto-return to available after a cooldown.
2. overloaded → available: after the cooldown (default 10 minutes), once the first successful response is confirmed.
3. available → retired: past the announced retirement date, or on a permanent not-found error. Never auto-reverted.
4. available → withdrawn: on behavior indicating withdrawal (a model that succeeded until moments ago starts getting rejected en masse with validation or permission errors). With no successor defined, scale down to the next in-role candidate. Reverted only by an explicit operator action.

The key is that retired and withdrawn are not auto-recovered. Only overload naturally heals over time; the other two won't return unless the outside world changes. If a machine reverts on its own, it keeps slamming a withdrawn model and mass-produces failures.

// availability-machine.ts
const OVERLOAD_COOLDOWN_MS = 10 * 60 * 1000; // 10 minutes
 
interface Health {
  state: Availability;
  overloadedAt?: number;
  consecutive429: number;
}
 
const health = new Map<string, Health>(); // key: model ID
 
export function noteResult(
  id: string,
  outcome: "ok" | "overloaded" | "retired" | "withdrawn",
) {
  const h = health.get(id) ?? { state: "available", consecutive429: 0 };
  if (outcome === "ok") {
    h.state = "available";
    h.consecutive429 = 0;
    h.overloadedAt = undefined;
  } else if (outcome === "overloaded") {
    h.consecutive429 += 1;
    if (h.consecutive429 >= 3) {
      h.state = "overloaded";
      h.overloadedAt = Date.now();
    }
  } else if (outcome === "retired") {
    h.state = "retired"; // no auto-recovery
  } else if (outcome === "withdrawn") {
    h.state = "withdrawn"; // operator action only
  }
  health.set(id, h);
}
 
export function effectiveState(id: string): Availability {
  const h = health.get(id);
  if (!h) return "available";
  // Only overload heals naturally over time.
  if (h.state === "overloaded" && h.overloadedAt &&
      Date.now() - h.overloadedAt > OVERLOAD_COOLDOWN_MS) {
    return "available"; // cooldown over; confirmed by the next success
  }
  return h.state;
}

Implement the Router

With the registry and health in place, the router becomes a thin layer that just "takes a logical role and returns the first usable candidate." The distinctions among withdrawal, retirement, and overload are all absorbed into effectiveState, so the router itself carries no branching.

// router.ts
import Anthropic from "@anthropic-ai/sdk";
import { REGISTRY, Role } from "./model-registry";
import { noteResult, effectiveState } from "./availability-machine";
 
const client = new Anthropic(); // ANTHROPIC_API_KEY from env
 
function pickModel(role: Role): string {
  const candidates = REGISTRY[role]
    .filter((m) => effectiveState(m.id) === "available");
  if (candidates.length === 0) {
    throw new Error(`no available model for role ${role}`);
  }
  return candidates[0].id;
}
 
function classifyError(err: unknown): "overloaded" | "retired" | "withdrawn" | "other" {
  const e = err as { status?: number; error?: { type?: string } };
  if (e.status === 429 || e.status === 529) return "overloaded";
  // A permanent not-found is treated as retirement.
  if (e.error?.type === "not_found_error") return "retired";
  // A model that succeeded until moments ago, now rejected on permission, hints at withdrawal.
  if (e.status === 403 || e.error?.type === "permission_error") return "withdrawn";
  return "other";
}
 
export async function route(role: Role, params: Omit<Anthropic.MessageCreateParams, "model">) {
  let lastErr: unknown;
  for (let attempt = 0; attempt < REGISTRY[role].length; attempt++) {
    const model = pickModel(role);
    try {
      const res = await client.messages.create({ ...params, model });
      noteResult(model, "ok");
      return { res, model };
    } catch (err) {
      lastErr = err;
      const kind = classifyError(err);
      if (kind === "other") throw err; // unexpected: rethrow
      noteResult(model, kind);
      // overloaded may recover after cooldown, but this call falls
      // through to the next candidate to return a response now.
    }
  }
  throw lastErr;
}

What pays off here is classifying the error kind up front with classifyError. A 403 or permission error can come from a misconfiguration, but when it starts suddenly on a model that succeeded until moments ago, it is worth treating as a withdrawal signal. If false positives worry you, add a guard: only flag withdrawn for a model that has a successful call within the last N minutes.

Detect Withdrawal Early With a Daily Preflight

Before the nightly batch runs, place a preflight that fires a single tiny call to each role's head candidate to catch withdrawal and retirement early. It is far cheaper and safer than discovering the problem mid-way through a heavy job.

# preflight.py — one lightweight probe per role
import anthropic
 
client = anthropic.Anthropic()  # ANTHROPIC_API_KEY from env
 
PROBE_BY_ROLE = {
    "deep": "claude-opus-4-8",
    "balanced": "claude-sonnet-4-6",
    "fast": "claude-haiku-4-5",
}
 
def probe(model_id: str) -> str:
    try:
        client.messages.create(
            model=model_id,
            max_tokens=1,
            messages=[{"role": "user", "content": "ok"}],
        )
        return "available"
    except anthropic.APIStatusError as e:
        if e.status_code in (429, 529):
            return "overloaded"   # transient; batch may proceed
        if e.status_code == 404:
            return "retired"      # switch to successor required
        if e.status_code == 403:
            return "withdrawn"    # scale-down decision required
        raise
 
if __name__ == "__main__":
    blocking = []
    for role, model_id in PROBE_BY_ROLE.items():
        state = probe(model_id)
        print(f"{role:9s} {model_id:20s} -> {state}")
        if state in ("retired", "withdrawn"):
            blocking.append((role, model_id, state))
    if blocking:
        for role, model_id, state in blocking:
            print(f"::alert:: role={role} model={model_id} state={state}")

A max_tokens=1 probe costs a trivial amount per call. In my own operation, one run a day across three roles came to a few yen a month. Slipping it in right before the nightly batch nearly eliminated the "discover the withdrawal mid-way through a heavy job" accident.

Don't Let Fallback Corrupt Your Cost Records

An easy thing to miss is cost accounting when a fallback changes the model. When the deep role downgrades from claude-opus-4-8 to claude-sonnet-4-6, the per-token price changes for the same token count. Keep charging at the first-choice price without recording the downgrade and your end-of-month totals quietly drift.

The router returns the model ID that actually succeeded, so always account using that return value.

// cost.ts
import { REGISTRY } from "./model-registry";
 
export function recordCost(
  usedModelId: string,
  inputTokens: number,
  outputTokens: number,
): number {
  const entry = Object.values(REGISTRY).flat().find((m) => m.id === usedModelId);
  if (!entry) throw new Error(`unknown model: ${usedModelId}`);
  const usd =
    (inputTokens / 1_000_000) * entry.inputPer1M +
    (outputTokens / 1_000_000) * entry.outputPer1M;
  // Price by the model actually used, not the first choice.
  return usd;
}

Downgrades can be cheaper, but a configuration where the fast role climbs to balanced under overload can also raise the unit price. Anchor accounting on the actually-used model so re-pricing is correct in either direction. In a period like today, when the billing scheme itself is under review, this "account by actual use" becomes your direct way of observing real cost.

Lessons and Pitfalls From Production

After running this setup for a few weeks, here is what I saw, candidly.

First, treating automatic withdrawal detection as advisory only turned out to be the practical stance. Declaring a 403 to be a withdrawal leaves too much room for false positives. I eventually settled on: when the preflight probe reports withdrawn, notify a human, and let the operator confirm the registry state change. The only things I let the machine do automatically are overload scale-down and recovery. The line between what a machine may decide in production and what a person should decide is best drawn by the cost of a false positive.

Second, always keep one in-role alternative that is not from the same lineage. In an event like today's withdrawal, where a whole family becomes unusable at once, escaping to another model in the same family is no guarantee of safety. Place a generation that is lower in capability but reliably present as the deep role's scale-down target, so the worst case is still "scale down and finish."

Third, set the overloaded cooldown too short and you get oscillation—repeatedly reverting to the first choice during a peak only to be rejected again. I started at 10 minutes and tuned it against the measured duration of 529 spells. I recommend measuring how long overload actually lasts before settling on a value, rather than hard-coding a guess.

Finally, the essence of this design comes down to clearly holding "things that return if you wait" and "things that won't return until the outside world changes" as distinct states in code. When a day like today arrives, being able to change one line in the registry—rather than scrambling to rewrite code—is what keeps automation running. For someone operating several systems alone, that quiet assurance is worth a great deal.

As a next step, count how many places in your own pipeline hard-code the model string you use today, with a quick grep. That number is exactly how many edits a generation change or a withdrawal will cost you.