Cancelling Claude API Streams the Right Way: AbortController, Token Billing, and Connection Hygiene

Python (asyncio): the same shape, just async with

The Python SDK exposes the cancellation mechanism through context managers and asyncio.CancelledError. Pair it with FastAPI's request.is_disconnected() and you get the same hygiene.

import asyncio
from anthropic import AsyncAnthropic
from fastapi import FastAPI, Request
from fastapi.responses import StreamingResponse
 
app = FastAPI()
client = AsyncAnthropic()
 
@app.post("/api/chat/stream")
async def stream_chat(request: Request, body: dict):
    output_tokens = 0
 
    async def generate():
        nonlocal output_tokens
        try:
            async with client.messages.stream(
                model="claude-sonnet-4-6",
                max_tokens=1024,
                messages=body["messages"],
            ) as stream:
                async for event in stream:
                    # Stop pulling if the client is gone
                    if await request.is_disconnected():
                        break
 
                    if event.type == "content_block_delta" and event.delta.type == "text_delta":
                        yield f"data: {event.delta.text}\n\n"
                    elif event.type == "message_delta":
                        output_tokens = event.usage.output_tokens
 
                yield "data: [DONE]\n\n"
        except asyncio.CancelledError:
            # FastAPI raises this when the connection is torn down
            pass
        finally:
            await record_usage(output_tokens, cancelled=await request.is_disconnected())
 
    return StreamingResponse(generate(), media_type="text/event-stream")
 
async def record_usage(output_tokens: int, cancelled: bool):
    # Persist to your metrics pipeline
    pass

await request.is_disconnected() polls the underlying TCP connection. Catch asyncio.CancelledError and let it pass — that's the user-cancelled path, not an exception worth alerting on.

The gotcha that took down our endpoint: connection-pool starvation

Here's the failure mode I learned the hard way. If you handle cancellation lazily, every aborted request leaves an HTTPS connection to Anthropic in a half-open state. The SDK's underlying connection pool fills up. Around 100 concurrent in-flight requests, new chats stop returning. You see ECONNRESET sprinkle into your logs and a slow climb in p99 latency before everything tips over. None of this is obviously about cancellation until you trace it back to half-open upstream connections that nobody is reading from. The fix is one-line, but you have to know to look for it.

Three rules that prevent this:

• Always cancel through the SDK's signal (or async with in Python). Hand-rolling fetch and trying to cancel() the underlying ReadableStream tends to leak — let the SDK do its cleanup.
• Wire client-disconnect detection with req.on("close") or request.is_disconnected(). Without it, the upstream stream keeps draining tokens long after the user has left.
• Set a hard timeout for safety. AbortSignal.timeout(60_000) covers the case where the connection is silently dead but no event fires.

If you want more depth on the failure modes here, Streaming pitfalls in the Claude API and field notes on recovering from partial streaming failures cover the runtime symptoms in detail.

How to verify the billing claim yourself

You don't have to take my word that "you only pay for what was generated." The Anthropic API console has a usage panel, and the SDK exposes usage in the final stream event, so you can prove this end to end in five minutes:

1. Send a deliberately long generation request (max_tokens: 4096) and abort after the first 20 tokens render.
2. In the abort handler, log the most recent event.usage.output_tokens you saw.
3. Wait a few minutes for usage aggregation, then open the console — the request you just made should show roughly the same output token count, plus a small drift.

If the console shows dramatically more than what you logged, that's a signal the abort wasn't actually closing the upstream connection — go back and check your req.on("close") wiring. I keep a small load test that does exactly this on every release of our chat service; it's caught two regressions in the past year, both times from middleware that was buffering the response body and preventing the close event from firing.

What the docs don't tell you: four things I only learned in production

Everything below came out of running a cancellable chat feature inside an iOS app I build and maintain on my own, on the App Store, with no team to catch my mistakes. None of it is in the documentation, and I doubt I would have found any of it by reading harder.

The cancellation drift is smaller than you fear

How many extra tokens does the server generate between your abort() and the socket actually closing? I measured it 100 times on Sonnet 4.6 with max_tokens: 1024, aborting once roughly 20 tokens had rendered.

At 2,400 cancellations a month that is roughly 28,800 extra output tokens. Priced at Sonnet 5's introductory rate of $10 per million output tokens (through 2026-08-31), you are looking at about $0.29 a month.

Drift is noise. Trading away a responsive Stop button to save it would be a bad deal, and I would not make it.

The real money is in the close handler you forgot to wire

Run the same 2,400 cancellations without req.on("close") and the server keeps generating an average of 780 output tokens per request that nobody will ever read (measured mean under max_tokens: 1024).

That is about 1.87 million tokens, or roughly $18.70 a month — 64 times the drift.

The billing question was never "am I charged when I cancel?" It was "does my cancellation actually reach the server?" Reframing it that way changed how I review every streaming endpoint I write.

Proxy buffering swallows the close event

Here is the one that cost me an afternoon.

With nginx sitting in front at its default proxy_buffering on, a client disconnect took up to 30 seconds to surface as a close event in Node. While the response body sits in the proxy's buffer, the upstream process still believes somebody is reading.

For any SSE endpoint, pair these two:

• At the proxy: proxy_buffering off;, or send X-Accel-Buffering: no on the response
• In the app: Cache-Control: no-cache, no-transform (drop no-transform and an intermediary may recompress, which re-introduces buffering)

Behind Cloudflare Workers, returning a TransformStream directly stays unbuffered — but the moment you await response.text() you are back in the same hole.

Your connection pool is smaller than the number you configured

Size the SDK pool for 100 concurrent streams, skip disconnect detection, and every abandoned stream holds its seat for the full 8.4 second average generation time. In my measurements, effective concurrency settled at 62.

You can paper over that by raising the pool from 100 to 160. It works, and it treats the symptom. One line of close wiring treats the cause.

Fold cancel, timeout, and shutdown into a single signal

Here is the shape I landed on. The trick is to compose three abort sources into one AbortSignal while still being able to recover which one fired — because that answer decides whether you may retry.

import Anthropic from "@anthropic-ai/sdk";
import type { Request, Response } from "express";
 
const client = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY! });
 
type StopReason = "completed" | "user_cancelled" | "timeout" | "shutdown" | "failed";
 
// One global signal so SIGTERM tears down every in-flight stream
const shutdown = new AbortController();
process.once("SIGTERM", () => shutdown.abort());
 
export async function streamChat(req: Request, res: Response) {
  const user = new AbortController();
  req.on("close", () => user.abort());
 
  const signal = AbortSignal.any([
    user.signal,
    AbortSignal.timeout(60_000),
    shutdown.signal,
  ]);
 
  res.setHeader("Content-Type", "text/event-stream");
  res.setHeader("Cache-Control", "no-cache, no-transform");
  res.setHeader("X-Accel-Buffering", "no"); // defeat nginx buffering
 
  let outputTokens = 0;
  let inputTokens = 0;
  let reason: StopReason = "completed";
 
  try {
    const stream = await client.messages.stream(
      { model: "claude-sonnet-4-6", max_tokens: 1024, messages: req.body.messages },
      { signal },
    );
 
    for await (const event of stream) {
      if (event.type === "message_start") {
        inputTokens = event.message.usage.input_tokens;
      } else if (event.type === "content_block_delta" && event.delta.type === "text_delta") {
        res.write(`data: ${JSON.stringify({ text: event.delta.text })}\n\n`);
      } else if (event.type === "message_delta") {
        outputTokens = event.usage.output_tokens; // always overwrite with the latest
      }
    }
    res.write("data: [DONE]\n\n");
    res.end();
  } catch (err) {
    reason = classify(err, user.signal, shutdown.signal);
    if (reason === "failed") {
      res.write(`data: ${JSON.stringify({ error: "stream_failed" })}\n\n`);
      res.end();
    }
  } finally {
    // Retry decisions read `reason`. user_cancelled and shutdown are never resent.
    await ledger.record({ inputTokens, outputTokens, reason, at: Date.now() });
  }
}
 
function classify(err: unknown, user: AbortSignal, shutdown: AbortSignal): StopReason {
  const aborted = err instanceof Error && err.name === "AbortError";
  if (!aborted) return "failed";
  if (user.aborted) return "user_cancelled";
  if (shutdown.aborted) return "shutdown";
  return "timeout"; // came from AbortSignal.timeout
}
 
const ledger = {
  async record(_row: { inputTokens: number; outputTokens: number; reason: StopReason; at: number }) {
    // Persist to Postgres or your metrics pipeline.
    // Bucketing by `reason` reproduces the drift table above for your own workload.
  },
};

AbortSignal.any() is available from Node.js 20 onward. Composing the three sources is easy; the part people skip is classify(). Lose the ability to tell a user cancel from a transient timeout and you will eventually resend a request the user deliberately stopped, and bill them twice for it.

Bucket outputTokens by reason and the drift table becomes something you own rather than something you borrowed from this article. If you want the wider cost picture, forecasting monthly token costs from the first three days pairs naturally with this ledger.

Handle each stop reason differently

"The stream ended early" covers four very different situations, and they want four different responses.

The rule I use when I am unsure: if a human stopped it, never resend. If a machine stopped it, resend. That single sentence prevents nearly every double-billing incident.

And if the work does not need to be real time at all, consider not streaming. Moving asynchronous jobs to the Messages Batches API for half the cost is frequently faster than perfecting your cancellation design.

Pre-production checklist

Six items, ordered by how much they matter.

1. req.on("close") (or request.is_disconnected()) is wired and calls abort()
2. A hard limit of roughly 60 seconds is in place via AbortSignal.timeout()
3. usage.output_tokens from message_delta is overwritten on every event and recorded in finally
4. AbortError is not logged at error level — it is a normal outcome, not a fault
5. The upstream proxy has proxy_buffering off / X-Accel-Buffering: no, and you have measured how long the close event takes to fire
6. If you wrap the SDK in your own retry logic, it checks signal.aborted before resending

Item 5 is the one you cannot verify by reading code. Close a browser tab against staging and watch the clock until Stream cancelled appears in the logs. That single observation is what surfaced the 30-second delay in my own setup.

Don't conflate cancellation with retry

One last subtle point. Cancellation (the user said stop) and retry (the SDK is automatically resending after a transient failure) look similar but must be handled differently.

• Cancellation: deliberate, must not be retried — retrying double-bills the user.
• Retry: triggered by 5xx responses or 429s with Retry-After. The SDK handles it with exponential backoff automatically.

The SDK does not retry after an AbortError, but if you've layered your own retry wrapper on top, make sure it checks controller.signal.aborted before resending. Otherwise a deliberate cancel becomes two billed responses. The retry side of the coin is covered in Retry-After header and backoff strategy.

What to do next

The smallest meaningful change you can make today is to add req.on("close", () => controller.abort()) (or request.is_disconnected() in Python) to your existing chat endpoint. Five minutes of work, and the long tail of orphaned streams from users who closed the tab quietly disappears. Your connection pool gets headroom you didn't know you'd lost, and your bill drops by however many tokens were getting generated into the void.

After that, the second priority is the load-test in the previous section: bake a "cancel after N tokens" check into your release pipeline. Connection-pool starvation is the kind of bug that doesn't show up in unit tests and only surfaces under real concurrency, so testing it in CI is the only way to keep the regressions out.