Claude API on Bun in Production: Migration Decisions and Implementation Patterns That Actually Survive Real Traffic

A streaming-aware HTTP server with Bun.serve

Bun.serve gives you a much friendlier API than Node's http.createServer. Here is how I forward a Claude streaming response straight to the client:

// src/server.ts
import Anthropic from "@anthropic-ai/sdk";
import { recordUsage } from "./usage";
import { checkRateLimit, releaseRateLimit } from "./ratelimit";
import { classifyError } from "./errors";
 
const client = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY! });
 
const server = Bun.serve({
  port: 8787,
  // Bun supports async fetch natively
  async fetch(req: Request) {
    const url = new URL(req.url);
    if (url.pathname !== "/api/chat") {
      return new Response("Not Found", { status: 404 });
    }
 
    const userId = req.headers.get("x-user-id") ?? "anonymous";
 
    // Allow at most two concurrent requests per user
    const allowed = await checkRateLimit(userId);
    if (!allowed) {
      return new Response("Too Many Requests", { status: 429 });
    }
 
    const { messages } = await req.json();
 
    const stream = new ReadableStream({
      async start(controller) {
        try {
          const response = await client.messages.stream({
            model: "claude-opus-4-6",
            max_tokens: 1024,
            messages,
          });
 
          for await (const event of response) {
            if (event.type === "content_block_delta") {
              const text = (event.delta as { text: string }).text;
              controller.enqueue(new TextEncoder().encode(text));
            }
          }
 
          const final = await response.finalMessage();
          recordUsage({
            userId,
            inputTokens: final.usage.input_tokens,
            outputTokens: final.usage.output_tokens,
            createdAt: Date.now(),
          });
 
          controller.close();
        } catch (err) {
          const classified = classifyError(err);
          console.error("[chat-error]", classified);
          controller.error(classified);
        } finally {
          releaseRateLimit(userId);
        }
      },
    });
 
    return new Response(stream, {
      headers: {
        "Content-Type": "text/plain; charset=utf-8",
        "Cache-Control": "no-cache, no-transform",
      },
    });
  },
});
 
console.log(`Listening on http://localhost:${server.port}`);

Note that messages.stream returns something you can iterate with for await directly. The Anthropic SDK supports the same shape on Node.js. Most of the total response time is upstream generation on Anthropic's side, so I would not expect the runtime to change it much. What Bun changes is startup speed and how lightly you carry many open connections.

A quick smoke test from the command line:

curl -N -X POST http://localhost:8787/api/chat \
  -H "Content-Type: application/json" \
  -H "x-user-id: dev1" \
  -d '{"messages":[{"role":"user","content":"Hello in 3 words"}]}'
# → Hello, hello, hello!

Tracking usage with the built-in SQLite

bun:sqlite gives you better-sqlite3-class performance with no external dependency. To keep an honest tally of what you owe Anthropic, persist input_tokens and output_tokens per request.

// src/usage.ts
import { Database } from "bun:sqlite";
 
const db = new Database("usage.db", { create: true });
 
db.exec(`
  CREATE TABLE IF NOT EXISTS usage_log (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    user_id TEXT NOT NULL,
    input_tokens INTEGER NOT NULL,
    output_tokens INTEGER NOT NULL,
    created_at INTEGER NOT NULL
  );
  CREATE INDEX IF NOT EXISTS idx_usage_user ON usage_log(user_id, created_at);
  PRAGMA journal_mode = WAL;
`);
 
const insertStmt = db.prepare(`
  INSERT INTO usage_log (user_id, input_tokens, output_tokens, created_at)
  VALUES ($userId, $inputTokens, $outputTokens, $createdAt)
`);
 
export interface UsageRecord {
  userId: string;
  inputTokens: number;
  outputTokens: number;
  createdAt: number;
}
 
export function recordUsage(rec: UsageRecord) {
  insertStmt.run({
    $userId: rec.userId,
    $inputTokens: rec.inputTokens,
    $outputTokens: rec.outputTokens,
    $createdAt: rec.createdAt,
  });
}
 
export function getDailyCost(userId: string, date: Date): number {
  const start = new Date(date);
  start.setHours(0, 0, 0, 0);
  const end = new Date(start);
  end.setDate(end.getDate() + 1);
 
  const row = db
    .prepare(
      `SELECT
         COALESCE(SUM(input_tokens), 0) AS in_tok,
         COALESCE(SUM(output_tokens), 0) AS out_tok
       FROM usage_log
       WHERE user_id = $userId AND created_at >= $start AND created_at < $end`
    )
    .get({
      $userId: userId,
      $start: start.getTime(),
      $end: end.getTime(),
    }) as { in_tok: number; out_tok: number };
 
  // Claude Opus 4.6 pricing (USD per million tokens: $5 input, $25 output — checked against the official pricing page)
  // Prices get revised; keep these in config rather than hard-coding them in production
  const inputCost = (row.in_tok / 1_000_000) * 5;
  const outputCost = (row.out_tok / 1_000_000) * 25;
  return inputCost + outputCost;
}

The prepared statement runs synchronously. Writing 20,000 rows one at a time with autocommit, I measured about 970 inserts per second with the default journal and about 5,600 per second in WAL mode (same sandbox as above). The same INSERT ran more than five times faster just by switching on WAL. For services where the Claude response itself takes seconds, either would do for a while, but the headroom is worth taking. Always enable WAL in production — PRAGMA journal_mode = WAL also removes the writer-blocks-readers behavior that bites smaller services first.

Concurrency control: a minimum-viable rate limiter

For edge-deployed services there is rarely a justification to introduce Redis. Here is a deliberately minimal in-memory limiter that uses only Bun primitives.

// src/ratelimit.ts
const inflight = new Map<string, number>();
const MAX_PARALLEL = 2;
 
export async function checkRateLimit(userId: string): Promise<boolean> {
  const current = inflight.get(userId) ?? 0;
  if (current >= MAX_PARALLEL) {
    return false;
  }
  inflight.set(userId, current + 1);
  return true;
}
 
export function releaseRateLimit(userId: string) {
  const current = inflight.get(userId) ?? 0;
  if (current <= 1) {
    inflight.delete(userId);
  } else {
    inflight.set(userId, current - 1);
  }
}

This is process-local. Once you scale beyond a single Bun process you will need either a SQLite-backed short-term table or Bun's built-in Redis client (Bun.redis). For a one-developer service serving up to a million requests per month, a single Bun process tends to be perfectly adequate.

Where you release the slot matters more than it looks. When I first wrote this section, I told readers to wrap the whole fetch handler in try...finally and call releaseRateLimit there. If you had wired it that way, the limit would never have fired.

The reason is simple: the fetch handler returns as soon as it has built the Response. The body keeps streaming after that. The finally runs before the stream has really started — so the slot is handed back almost immediately.

I checked it by swapping the upstream for a mock that emits five chunks, 200 ms apart, and firing four simultaneous requests from the same user (the limit is two).

// rl.ts — comparing where to release the slot (mock upstream)
const MAX = 2;
const inflight = new Map<string, number>();
const acquire = (u: string) => {
  const c = inflight.get(u) ?? 0;
  if (c >= MAX) return false;
  inflight.set(u, c + 1);
  return true;
};
const release = (u: string) => {
  const c = inflight.get(u) ?? 0;
  c <= 1 ? inflight.delete(u) : inflight.set(u, c - 1);
};
async function* upstream() {
  for (let i = 0; i < 5; i++) { await Bun.sleep(200); yield "x"; }
}
 
function handler(mode: "fetch-finally" | "stream-finally") {
  return async () => {
    if (!acquire("u")) return new Response("429", { status: 429 });
    const make = () =>
      new Response(new ReadableStream({
        async start(c) {
          try {
            for await (const t of upstream()) c.enqueue(new TextEncoder().encode(t));
            c.close();
          } finally {
            if (mode === "stream-finally") release("u");
          }
        },
      }));
    if (mode === "fetch-finally") { try { return make(); } finally { release("u"); } }
    return make();
  };
}
 
for (const mode of ["fetch-finally", "stream-finally"] as const) {
  const server = Bun.serve({ port: 0, fetch: handler(mode) });
  const codes = await Promise.all([1, 2, 3, 4].map(async () => {
    const r = await fetch(`http://localhost:${server.port}/`);
    await r.text();
    return r.status;
  }));
  console.log(mode, codes.join(" "));
  server.stop(true);
}

Results on Bun 1.4.2:

With the release in the fetch handler, a limit of two let all four through, with four streams hanging off the upstream at once. With the release inside the stream, two requests were rejected with 429 and the upstream never saw more than two. That is why server.ts above releases inside start.

Count until the last byte has gone out, not until the handler returns.

Error classification: building dashboards that mean something

Logging Anthropic errors with a raw console.error is not enough to operate the service. Classifying by status and category makes it possible to immediately see "are we being rate limited", "are we paying for an outage upstream", or "is the client misconfigured".

// src/errors.ts
import Anthropic from "@anthropic-ai/sdk";
 
export type ErrorCategory =
  | "client_error" // 400/422 — bad request
  | "auth_error"   // 401/403 — credentials
  | "rate_limit"   // 429 — Anthropic rate limit
  | "overloaded"   // 529 — Anthropic overloaded
  | "server_error" // 500/503
  | "network"      // timeout / disconnect
  | "client_abort" // the client hung up (we stopped it ourselves)
  | "unknown";
 
export interface ClassifiedError {
  category: ErrorCategory;
  status?: number;
  message: string;
  shouldRetry: boolean;
}
 
export function classifyError(err: unknown): ClassifiedError {
  // APIUserAbortError extends APIError and carries no status.
  // Check it first, or it falls through to client_error below.
  if (err instanceof Anthropic.APIUserAbortError) {
    return { category: "client_abort", message: err.message, shouldRetry: false };
  }
 
  if (err instanceof Anthropic.APIError) {
    const status = err.status;
    if (status === 429) {
      return { category: "rate_limit", status, message: err.message, shouldRetry: true };
    }
    if (status === 529) {
      return { category: "overloaded", status, message: err.message, shouldRetry: true };
    }
    if (status === 401 || status === 403) {
      return { category: "auth_error", status, message: err.message, shouldRetry: false };
    }
    if (status !== undefined && status >= 500) {
      return { category: "server_error", status, message: err.message, shouldRetry: true };
    }
    return { category: "client_error", status, message: err.message, shouldRetry: false };
  }
 
  if (err instanceof Anthropic.APIConnectionError) {
    return { category: "network", message: err.message, shouldRetry: true };
  }
 
  return {
    category: "unknown",
    message: err instanceof Error ? err.message : String(err),
    shouldRetry: false,
  };
}

Once you record the category you can wire SLO-friendly alerts: "page on the first auth_error", "page only when rate_limit exceeds 10 in 5 minutes", and so on. The category vocabulary is what makes those rules expressible.

I split out client_abort for a reason. When I cut the client mid-stream against a mock Anthropic server, the SDK threw APIUserAbortError. It has no status, so the classifier before this change filed it under client_error ("bad request"). Every disconnect would have inflated that bucket and buried the real 400s.

Stopping the meter when the client hangs up

If the client disconnects but the upstream call keeps streaming, the output tokens it generates still land on your bill. Bun's Request.signal fires on client disconnect, so wire it through to the SDK's AbortController.

// In the fetch handler, before creating the stream
const ac = new AbortController();
req.signal.addEventListener("abort", () => {
  console.log("[client-disconnect]", { userId });
  ac.abort();
});
 
const response = await client.messages.stream(
  {
    model: "claude-opus-4-6",
    max_tokens: 1024,
    messages,
  },
  { signal: ac.signal }
);

I ran this rather than assume it. Against a mock Anthropic server (eight chunks, 150 ms apart), I cut the client after two chunks. req.signal aborted, the stream's cancel fired, and the SDK raised APIUserAbortError, in that order — and the mock's own stream was closed. Calling controller.error() after cancel does not throw. When the client read to the end instead, finalMessage() returned the usage (the mock's 10 input / 8 output tokens) and it was recorded.

Five Bun-specific traps to know about up front

Before you take any of this to production, here are the issues I have hit in real systems, ranked by how often they bit me.

Node compatibility edges. Bun's Node compatibility is high but not 100%. Some crypto.createHash variants, low-level net operations, and worker_threads can behave subtly differently. The Anthropic SDK itself runs cleanly, but old transitive HTTP clients sometimes do not. Also worth knowing: Bun does not run dependencies' install scripts by default. After adding a package with native code, check bun pm untrusted for scripts that were held back and allow only the ones you need with bun pm trust <package>. Going one by one is safer than --all.

Test runner differences. bun test aims for Jest API parity, but describe.skip.each and parts of snapshot matching diverge. Migrating an existing Jest suite verbatim can produce failures. My recommendation: write new projects against bun test from the start, and for migrations keep the existing Jest suite running on Node while you add new tests with bun test.

// tests/proxy.test.ts — native bun test
import { test, expect } from "bun:test";
import { classifyError } from "../src/errors";
import Anthropic from "@anthropic-ai/sdk";
 
test("classifies 429 as rate_limit with retry", () => {
  const err = new Anthropic.APIError(429, undefined, "rate limited", undefined);
  const result = classifyError(err);
  expect(result.category).toBe("rate_limit");
  expect(result.shouldRetry).toBe(true);
});
 
test("classifies 401 as auth_error without retry", () => {
  const err = new Anthropic.APIError(401, undefined, "unauthorized", undefined);
  const result = classifyError(err);
  expect(result.category).toBe("auth_error");
  expect(result.shouldRetry).toBe(false);
});

Linux binary size. bun build --compile produces a single binary, but it bundles the whole runtime. A minimal HTTP server came out at 78 MB on Linux x86_64. That is over Lambda's 50 MB direct-upload limit. The pragmatic answer is container images.

# Dockerfile
# In production, pin this to the tag you actually tested
FROM oven/bun:1-alpine AS base
WORKDIR /app
COPY package.json bun.lock ./
RUN bun install --frozen-lockfile
 
COPY . .
EXPOSE 8787
CMD ["bun", "run", "src/server.ts"]

Memory model differences. Bun does not honor --max-old-space-size the way Node does. When you suspect a leak in a long-running Bun service, compare memory over time with and without bun --smol to narrow it down.

PM2 / forever quirks. Some process managers fail to detect bun cleanly. I have settled on Docker --restart=always or systemd with Type=simple for VM deployments.

Migration strategy: how I roll Node services onto Bun

If you already have a Node Claude API service in production, swapping the runtime overnight is rightly scary. Here is the staged approach I have used across multiple projects.

Step one is to switch local development to Bun first. Change your package.json to bun src/server.ts --watch for scripts.dev. Production stays on Node, but every developer gets faster boots and instant TypeScript. This alone usually pays for the migration cost in a week.

Step two is moving CI tests to Bun. bun test starts lightly, so the more tests you have, the more it shows. Compare CI wall-clock before and after. If a test fails, move just that one back to Jest on Node to triangulate. It is much cheaper to fail in CI than in production.

Step three is the runtime swap. Stand the Bun build up in staging for a week or two. Compare CPU, memory, p95 / p99 latency, and error rates side by side with the Node version using whatever you already have — Datadog, Cloudflare Analytics, an in-house dashboard, anything. "It feels faster" is not a strong enough signal to switch a runtime in production.

Four numbers are enough to decide on: p95 / p99 latency, memory at idle and under load, cold start, and error rate. Until I have those numbers in hand, I do not change the production runtime. That is the line I hold.

When switching actually makes things cheaper

Numbers beat vibes, so here is what I could measure. With the minimal server above, resident memory at the first response was about 54 MB on Node.js 22.22 and about 15 MB on Bun 1.4.2. A Claude proxy carries the SDK and SQLite on top, so the gap narrows. There is still room for more concurrent streams per instance.

I would not generalize that into "Bun is cheaper", though. If the bill drops, it drops because one instance became enough, not because Bun is intrinsically cheaper to host. A Claude proxy is bound less by CPU than by how many streams are open at once and how much memory each one holds. The estimate looks like this:

If your service already runs comfortably on one instance, expect the direct cost delta to be small. The bigger value is usually indirect: no TypeScript build step, faster CI, a cleaner dependency graph.

Before writing "it got cheaper", check whether the instance count actually went down. A number that did not come from your own load test is a weak basis for the decision.

What I assumed differed between Node and Bun, and what really did

The Anthropic SDK uses the global fetch on both runtimes. Node.js's fetch is built on undici; Bun ships its own implementation. For ordinary requests they look identical, and I checked the differences I assumed before writing them down.

The first is header name casing. My earlier draft said that reading headers.get("Anthropic-Request-Id") with a capital letter would silently return null on Bun. When I tried it, headers.get("Request-Id") and headers.get("request-id") returned the same value. Headers is case-insensitive by specification. The only difference is that enumerating with [...headers.keys()] yields lowercase names. Only code that enumerates headers to compare or store them needs to assume lowercase.

The second is Server-Sent Events chunk boundaries. The Anthropic SDK does its own parsing, so this rarely matters in practice. But if you write a custom streaming consumer, the chunk sizes from Response.body.getReader() can differ between runtimes. Do not split on a bare \n\n; buffer until you see a complete event boundary, and never assume one chunk equals one event.

Idle connection reuse might differ too, but I have not been able to measure it. What I have not measured, I write down as neither different nor the same.

Before I write that two runtimes differ, I check it in one line of code. Knowing the real differences keeps a working Node test from flaking on Bun without explanation. Not inventing differences is just as important.

Production observability without bringing in a heavy stack

For solo developers and small teams, full Datadog or New Relic deployments are often overkill. Bun's introspection primitives, combined with structured logging, get you most of what you need.

// src/observability.ts
type LogLevel = "info" | "warn" | "error";
 
interface LogFields {
  msg: string;
  userId?: string;
  reqId?: string;
  tokenUsage?: { input: number; output: number };
  latencyMs?: number;
  errorCategory?: string;
  [key: string]: unknown;
}
 
export function log(level: LogLevel, fields: LogFields) {
  const entry = {
    ts: new Date().toISOString(),
    level,
    ...fields,
  };
  // structured JSON to stdout — pickup-compatible with Cloud Run, Fly, Datadog Agent, etc.
  process.stdout.write(JSON.stringify(entry) + "\n");
}
 
export function startSpan(name: string) {
  const startedAt = performance.now();
  return {
    end(extra: Partial<LogFields> = {}) {
      const latencyMs = Math.round(performance.now() - startedAt);
      log("info", { msg: `span_end:${name}`, latencyMs, ...extra });
    },
  };
}

Wrap your fetch handler with startSpan("chat_request") and call span.end({ userId, tokenUsage }) in the finally block. That alone gives you per-request latency, token usage, and a request ID you can trace through your platform's log search. When you outgrow this you can graduate to OpenTelemetry without changing the call sites — just swap the implementation of log and startSpan.

The point is that you can ship a respectable observability story on Bun without adding heavyweight dependencies on day one. Premature instrumentation has its own costs.

Hot reload and developer ergonomics

The other unsung win of Bun is the developer feedback loop. bun --hot src/server.ts does live module replacement without restarting the process, which means your in-memory rate limiter and SQLite connection survive the reload. With Node and nodemon, every file save costs you a fresh process boot — typically two to three seconds — and forces you to re-establish any in-memory state.

For a Claude API service that's especially nice because you can keep an open streaming session in a browser tab while editing the prompt logic. The change re-applies, the next request uses the new code, and you don't lose your conversation context. Without the restart wait, you simply end up trying more variations.

// package.json
{
  "name": "claude-bun-proxy",
  "type": "module",
  "scripts": {
    "dev": "bun --hot src/server.ts",
    "test": "bun test",
    "build": "bun build src/server.ts --target=bun --outdir dist"
  },
  "dependencies": {
    "@anthropic-ai/sdk": "^0.133.0"
  }
}

A note on bun build: it produces a single file but does not bundle the runtime. The output is intended for Bun, not Node, which is precisely why the file size stays small. If you want a single binary for distribution, use bun build --compile, accept the 60–80 MB payload, and ship it inside a container.

When to reach for worker threads

For most Claude API proxies you do not need worker threads. The bulk of the latency is upstream, not local CPU. But there are two scenarios where they earn their keep on Bun.

The first is post-processing of streamed output. If you are running a regex-heavy moderation pass, computing a summary embedding, or doing structured extraction on the full response, you want that work off the request thread. Bun supports the standard worker_threads API, so the patterns you know from Node carry over.

The second is batched prompt formatting. If your service builds long prompts by joining many records (for example, a "summarize these 100 customer emails" feature), the JSON parsing and string concatenation can compete with the streaming loop for CPU. Pushing that work into a worker keeps your event loop responsive and your stream chunks moving.

A small caveat: cross-thread bun:sqlite connections do not share state. Each worker should open its own database handle, and you should rely on SQLite's own concurrency model (WAL mode helps here too) rather than passing the handle around.

Edge deployment options compared

If you are choosing where to host the Bun service, here is how the main options stack up in 2026 from a Claude API perspective.

• Cloud Run on GCP: most straightforward. Scaling-to-zero saves money, and request-based pricing matches Anthropic's billing model nicely.
• Fly.io: closest to "VM with a friendly dashboard". Best when you want long-lived processes (always-on rate-limit state) and predictable pricing.
• AWS App Runner: works but has rougher edges around cold start and image cache eviction. Reasonable if you are already in AWS.
• Cloudflare Workers: not a fit for raw Bun servers. Use Workers if you want the Cloudflare runtime; do not try to run Bun.serve inside a Worker.
• Self-hosted VM with systemd: still my preference for personal projects where I want predictable cost and full control.

The right choice usually depends on what else you are running, not on what is "best for Bun" in isolation. If you are already on Cloud Run, stay on Cloud Run.

Pre-launch checklist

Before pushing a Bun-based Claude API service into production, walk through this list:

• SQLite is in WAL mode, and you have load-tested that writes do not stall under realistic concurrency
• Request.signal actually fires on client disconnect (test it: curl -N, then Ctrl+C, then read the logs)
• Install scripts held back by Bun have been reviewed with bun pm untrusted, and only the needed ones allowed
• The Anthropic API key is injected via environment variables, not baked into the image
• Logs are structured JSON so Cloud Run, Cloudflare Logs, or Datadog can query them
• The service has run for 24 hours straight without --smol and shown no memory creep
• Staging has been compared head-to-head with the Node version for a week of real traffic
• Retry logic is in place for rate_limit and overloaded, with exponential backoff
• Against a mock upstream, firing the limit plus two simultaneous requests from one user returns 429 for the overflow
• Cutting the client mid-stream leaves client_abort in the logs and does not grow client_error

If you want to deepen the testing and rollout side, the Claude Code TDD workflow guide pairs nicely with the Bun-native test runner discussion above. To shore up streaming reliability specifically, the Claude API streaming disconnection recovery guide covers the failure modes that hit you during, not before, deployment.

What to do next

Pick the smallest service you currently run on Node, change one line in package.json to invoke bun src/server.ts, and try a normal day of development on it. In most projects that is the entire local migration. From there, move CI to bun test, then run a Bun build in staging next to the Node build for a week before flipping production.

You do not need to commit to "all Bun, everywhere". Even running Bun only in development, only in CI, or only on a single service tightens dependencies, shrinks images, and improves boot time. The right reason to adopt Bun is not "it is fast"; it is "the resulting system is simpler". That framing gives you an honest answer to whether the migration is worth your time.