Designing Graceful Degradation for the Claude API — A Four-Tier Fallback Architecture That Keeps AI Features Quietly Alive

Three SLIs that drive the tier switch

To switch tiers mechanically, you need to decide ahead of time what to measure. I aggregate the following three Service Level Indicators over a 1-minute window and feed them into a small decision engine:

1. p99 latency: 99th percentile latency of Claude API calls in the last minute. Threshold 800 ms triggers a candidate step-down.
2. 5xx rate: Share of 5xx responses in the same window. Threshold 2 percent triggers an immediate step-down.
3. 429 rate: Share of 429 responses in the same window. Threshold 5 percent triggers a candidate step-down. If 429 and 5xx are both elevated, step down by two tiers at once.

These signals should not be read independently. Bundle them into an SLO burn rate. A 14.4x burn rate over a one-hour window means you are burning an hour of monthly budget in real time and should demote immediately. This adapts the Google SRE Book burn-rate philosophy to AI features, and it has aligned well with what I have actually seen in production.

Decision engine in Python

What this code solves: given the latest 1-minute SLI snapshot, it returns the tier to operate in next and an audit-friendly reason string. The function is intentionally pure so the same logic can run inside a Cloudflare Workers Durable Object, an AWS Lambda fronted by DynamoDB Streams, or a plain cron job.

# tier_decider.py
from dataclasses import dataclass
from enum import IntEnum
from typing import Literal
 
class Tier(IntEnum):
    T0_FULL = 0
    T1_REDUCED = 1
    T2_CACHED = 2
    T3_STATIC = 3
 
@dataclass(frozen=True)
class SLISnapshot:
    """Last 1-minute SLI snapshot. Fed by Prometheus or CloudWatch upstream."""
    p99_latency_ms: float
    err_5xx_rate: float
    err_429_rate: float
    burn_rate_1h: float  # ratio against monthly SLO target
 
def decide_tier(
    current: Tier,
    sli: SLISnapshot,
    feature_priority: Literal["critical", "standard", "optional"] = "standard",
) -> tuple[Tier, str]:
    """Return the next tier and a human-readable reason."""
 
    # 1. SLO burn rate over 14.4x -> jump to T2 or T3
    if sli.burn_rate_1h >= 14.4:
        if feature_priority == "optional":
            return Tier.T3_STATIC, f"burn_rate={sli.burn_rate_1h:.1f} optional -> T3"
        return Tier.T2_CACHED, f"burn_rate={sli.burn_rate_1h:.1f} -> T2"
 
    # 2. 5xx rate above 2% -> step down once
    if sli.err_5xx_rate >= 0.02:
        return _step_down(current, reason=f"5xx={sli.err_5xx_rate:.1%}")
 
    # 3. 429 rate above 5% -> step down once
    if sli.err_429_rate >= 0.05:
        return _step_down(current, reason=f"429={sli.err_429_rate:.1%}")
 
    # 4. p99 latency above 800 ms -> step down once
    if sli.p99_latency_ms >= 800.0:
        return _step_down(current, reason=f"p99={sli.p99_latency_ms:.0f}ms")
 
    # 5. All signals healthy -> step up by one (the caller handles 30s hysteresis)
    if (
        sli.p99_latency_ms < 500.0
        and sli.err_5xx_rate < 0.005
        and sli.err_429_rate < 0.01
        and sli.burn_rate_1h < 2.0
    ):
        return _step_up(current, reason="all SLIs healthy")
 
    return current, "no change"
 
def _step_down(current: Tier, reason: str) -> tuple[Tier, str]:
    next_tier = Tier(min(current.value + 1, Tier.T3_STATIC.value))
    return next_tier, f"step_down to {next_tier.name}: {reason}"
 
def _step_up(current: Tier, reason: str) -> tuple[Tier, str]:
    next_tier = Tier(max(current.value - 1, Tier.T0_FULL.value))
    return next_tier, f"step_up to {next_tier.name}: {reason}"

The reason this engine moves only one tier at a time is empirical. If recovery jumps you from T3 directly back to T0, the moment the underlying issue twitches you slam into the same wall again. The caller in my deployment waits 30 seconds of continuous healthy readings before stepping up.

TypeScript wrapper for the call site

The decision engine produces a tier number. The client wrapper turns that number into actual behavior on each request. The version I run on Cloudflare Workers as a BFF looks like this:

// claude-degraded-client.ts
import Anthropic from "@anthropic-ai/sdk";
 
type Tier = 0 | 1 | 2 | 3;
 
interface DegradedClientOptions {
  apiKey: string;
  cacheGet: (key: string) => Promise<string | null>;
  cachePut: (key: string, value: string, ttlSec: number) => Promise<void>;
  currentTier: () => Promise<Tier>; // value comes from tier_decider
  featurePriority: "critical" | "standard" | "optional";
}
 
const MODEL_BY_TIER: Record<0 | 1, string> = {
  0: "claude-sonnet-4-6",
  1: "claude-haiku-4-5-20251001",
};
 
export class ClaudeDegradedClient {
  private readonly anthropic: Anthropic;
 
  constructor(private readonly opts: DegradedClientOptions) {
    this.anthropic = new Anthropic({ apiKey: opts.apiKey });
  }
 
  async respond(
    prompt: string,
    cacheKey: string,
    staticFallback: string,
  ): Promise<{ text: string; tierUsed: Tier; cached: boolean }> {
    const tier = await this.opts.currentTier();
 
    // Tier 2: prefer semantic cache
    if (tier >= 2) {
      const hit = await this.opts.cacheGet(cacheKey);
      if (hit) return { text: hit, tierUsed: 2, cached: true };
      if (tier === 3) {
        // T3: do not call the API at all
        return { text: staticFallback, tierUsed: 3, cached: false };
      }
      // T2 cache miss: regenerate with T1, then write back
      const text = await this.callModel(prompt, 1, /*thinking=*/ false);
      await this.opts.cachePut(cacheKey, text, 3600);
      return { text, tierUsed: 2, cached: false };
    }
 
    // Tier 0 / 1: ordinary Claude API call
    const useThinking = tier === 0 && this.opts.featurePriority === "critical";
    const text = await this.callModel(prompt, tier, useThinking);
 
    // Write successful responses into the cache for next time
    await this.opts.cachePut(cacheKey, text, 600);
    return { text, tierUsed: tier, cached: false };
  }
 
  private async callModel(
    prompt: string,
    tier: 0 | 1,
    thinking: boolean,
  ): Promise<string> {
    const model = MODEL_BY_TIER[tier];
    const msg = await this.anthropic.messages.create({
      model,
      max_tokens: tier === 0 ? 2048 : 1024,
      messages: [{ role: "user", content: prompt }],
      ...(thinking
        ? { thinking: { type: "enabled", budget_tokens: 4096 } }
        : {}),
    });
    return msg.content
      .filter((b) => b.type === "text")
      .map((b) => (b as { text: string }).text)
      .join("");
  }
}

The piece of this design that survives contact with real traffic is "T2 means cache-or-regenerate-then-cache," not "T2 means error on miss." If T2 starts cold and you reject misses, every user is treated like an outage. Regenerating once at the cheaper tier and writing back gives T2 a graceful warm-up curve that matches what humans actually perceive.

Fail-closed versus fail-open as a per-feature decision

When you finally reach T3, you have to decide whether to fail-closed (refuse the feature outright) or fail-open (serve a simple static response and continue). My working rules:

• Fail-closed for write-to-user-data features: auto-summarizing chat history, editing AI-managed lists, generating post-purchase confirmation text. A wrong answer here can corrupt the user's state. Refuse explicitly.
• Fail-open for display-only features: related-link blurbs, conversational search summaries, today's recommendation copy. A flat static line is acceptable because the broader UX still works.

The wallpaper apps in the Dolice portfolio see roughly one million requests per day. With AdMob revenue tied directly to availability, a blank screen is the option of last resort. Most feature failures degrade quietly into static lines that users do not even notice. Over twelve years there have been only a handful of explicit complaints. Write-to-user-data features, by contrast, fail-closed without apology: "this feature is temporarily unavailable, please try again later" is much safer than a wrong write.

Putting chaos drills into CI

A degradation design is only trustworthy if you exercise it on a schedule. I run a weekly chaos drill in a Cloudflare Workers preview environment from GitHub Actions:

# .github/workflows/chaos-degradation-drill.yml
name: chaos-degradation-drill
on:
  schedule:
    - cron: "0 18 * * 3"   # every Wednesday 03:00 JST
  workflow_dispatch:
 
jobs:
  drill:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: "22"
      - name: Inject synthetic 429 storm
        run: |
          node scripts/inject-fault.mjs \
            --target=preview \
            --pattern=429-storm \
            --duration=180
 
      - name: Verify tier descended to T2 or below
        run: |
          node scripts/assert-tier.mjs --expected-max=2 --within=60s
 
      - name: Verify cached responses still served
        run: |
          node scripts/assert-cache-hit-rate.mjs --min=0.8
 
      - name: Recovery within 5 minutes
        run: |
          node scripts/wait-tier.mjs --expected=0 --within=300s
 
      - name: Post drill report to ops channel
        if: always()
        run: |
          node scripts/post-drill-report.mjs --to=ops --result="${{ job.status }}"

The first time I ran this drill, it surfaced a silly bug: the code had stepped down to T2, but the tier_used metric stuck at the old value for over thirty seconds because the Durable Object write had not propagated to the dashboard. Without the drill, I would have discovered that for the first time during a real production incident.

Five operational gotchas

• Tier flap without hysteresis: instant step-up after recovery causes T0 to T2 oscillation every minute. Requiring 30 seconds of continuous healthy readings before stepping up eliminates the flap.
• Cache TTL too short collapses T2: a 60-second TTL pushes T2 into mostly-miss territory under load. I use 10 minutes for hot keys and 60 minutes for cold per-user keys.
• Static fallback text needs to be respectable: "data fetch failed" reads like a bug. "Today's recommendation is loading, please try again shortly" preserves trust during T3.
• Alert on the tier transition itself: alerting on T1, T2, T3 entries individually is more intuitive than alerting on raw SLI thresholds. I use emoji per tier in Slack so the on-call (me) can read the timeline at a glance.
• Tie cost telemetry to tier: plotting time spent in T1 against AdMob revenue over a week tells you which features can tolerate T2 for how long. In my operation this analysis ended up informing how I priced the Premium plan.

Why this design fits indie development

Closing thoughts on why the four-tier model is a particularly good fit for solo operators. I run the four Lab sites, the wallpaper app business, and my contemporary art practice in parallel as one person, so this perspective is real.

• During incidents, there is no one to page. The value of a system that quietly demotes itself instead of failing loudly is therefore disproportionately high.
• AdMob revenue varies sharply by feature. The four-tier map lines up directly with "what must not stop" versus "what we are willing to lose." Indie economics make the conversation simpler, not harder.
• Predictable monthly spend matters when you are personally on the hook for it. SLO burn-rate-driven demotion is a safety valve that prevents a bad week from blowing up the AI budget.

The reason the Dolice app business has survived twelve years and 50 million cumulative downloads is not flashy features. It is the quiet accumulation of designs like this one, where failure stays small and recovery happens without anyone watching. Bringing the Claude API into that stack has not changed the principle, only the surface area to which it applies. I hope this design serves anyone else operating an AI feature alone.