Claude API Messages Batches: Cutting Production Costs by Up to 50% with Async Processing

Constraints and When to Use Batch Processing

Before diving into implementation, understand the constraints to ensure you're using the right tool for the job.

Key Limitations

Volume and size limits

• 100,000 requests per batch, or 256 MB, whichever you hit first
• Oversized payloads come back as a 413 request_too_large
• I previously wrote that there's a per-account cap of "typically 20" concurrent batches. I couldn't source that number, so I've withdrawn it

Processing time and retention

• Requests that don't complete within 24 hours come back as expired
• Results stay downloadable for 29 days from batch creation — more on that below
• No real-time response, by design

Model availability

• "All active models support the Message Batches API." Model-by-model compatibility isn't something you need to check
• The failure mode to worry about is the opposite one: a retired model ID left behind in code. Keep model IDs in one constant

Only three parameters are rejected

• stream: true — results come back as a single file, not a stream
• speed — Fast mode tunes synchronous latency, which doesn't apply here
• max_tokens: 0 — a cache pre-warm would expire before the batch finished running

Include any of them and you get a validation error. Everything else goes through: vision, tool use including server tools, system messages, multi-turn conversations, extended thinking, and most beta features. My earlier warning about "beta feature restrictions" was vaguer and more pessimistic than the reality.

Best-Fit Use Cases

Batch processing shines for workloads where:

• Bulk document processing: Summarizing, classifying, or extracting data from hundreds to millions of documents
• Product catalog enrichment: Generating descriptions, tags, or SEO content for e-commerce inventories
• Content moderation: Screening large volumes of user-generated content offline
• Translation pipelines: Rendering multilingual versions of articles, product pages, or documentation
• Nightly reports: Generating analytics summaries, business intelligence reports, or digest emails
• Training data generation: Creating annotated datasets, synthetic examples, or evaluation sets

Batch processing is a poor fit when:

• Users are actively waiting for a response (chatbots, interactive tools)
• Later requests depend on earlier results (sequential reasoning chains)
• You need guaranteed sub-second latency

Step 1: Creating a Batch

Basic Implementation with the Python SDK

import anthropic
from anthropic.types.message_create_params import MessageCreateParamsNonStreaming
from anthropic.types.messages.batch_create_params import Request
 
client = anthropic.Anthropic(api_key="YOUR_API_KEY")
 
documents = [
    {"id": "doc_001", "content": "A detailed document about AI ethics..."},
    {"id": "doc_002", "content": "A technical paper on machine learning..."},
    {"id": "doc_003", "content": "An analysis of startup funding trends..."},
]
 
MODEL = "claude-sonnet-4-6"  # keep model IDs in one place
 
batch_requests = [
    Request(
        # custom_id: 1-64 chars, alphanumerics, hyphens and underscores only
        custom_id=doc["id"],
        params=MessageCreateParamsNonStreaming(
            model=MODEL,
            max_tokens=500,  # batches require at least 1; 0 is rejected
            messages=[
                {
                    "role": "user",
                    "content": f"Summarize the following document in 3 sentences:\n\n{doc['content']}",
                }
            ],
        ),
    )
    for doc in documents
]
 
batch = client.messages.batches.create(requests=batch_requests)
 
print(f"Batch ID: {batch.id}")
print(f"Status: {batch.processing_status}")
print(f"Pending: {batch.request_counts.processing}")
# Output:
# Batch ID: msgbatch_01HxxxxxxxxxxxxxxxxxxXXXX
# Status: in_progress
# Pending: 3

Note the call surface: client.messages.batches. You only need client.beta.messages.batches when you're attaching a beta header such as output-300k-2026-03-24 to raise the max_tokens ceiling. This article originally used the beta namespace throughout, and I've moved it to match the current docs.

The custom_id field is the important part. The docs are explicit that results can come back in any order, so the mapping between results and source data has to go through custom_id — never through array position. Use identifiers that mean something to you: database record IDs, file names, whatever you'll want to see in a log six weeks from now.

Step 2: Polling for Completion

After submitting a batch, you need a reliable polling strategy. This is where production implementations often differ from quick scripts.

Basic Polling Implementation

import time
import anthropic
 
def wait_for_batch_completion(
    client: anthropic.Anthropic,
    batch_id: str,
    poll_interval_seconds: int = 60,
    max_wait_minutes: int = 120,
) -> anthropic.types.messages.MessageBatch:
    """
    Wait for a batch to finish processing.
 
    Args:
        client: Anthropic client instance
        batch_id: The batch to wait for
        poll_interval_seconds: How often to check status
        max_wait_minutes: Maximum time to wait before raising TimeoutError
 
    Returns:
        Completed batch object
 
    Raises:
        TimeoutError: If max_wait_minutes is exceeded
        RuntimeError: If the batch is canceled
    """
    max_attempts = (max_wait_minutes * 60) // poll_interval_seconds
 
    for attempt in range(max_attempts):
        batch = client.messages.batches.retrieve(batch_id)
        status = batch.processing_status
 
        if status == "ended":
            print(f"Batch completed: {batch_id}")
            print(f"  Succeeded: {batch.request_counts.succeeded}")
            print(f"  Errored: {batch.request_counts.errored}")
            print(f"  Expired: {batch.request_counts.expired}")
            return batch
 
        elif status == "canceling":
            raise RuntimeError(f"Batch is being canceled: {batch_id}")
 
        else:
            # Still in_progress — keep waiting
            remaining = batch.request_counts.processing
            print(f"[{attempt + 1}/{max_attempts}] Processing... ({remaining} remaining)")
            time.sleep(poll_interval_seconds)
 
    raise TimeoutError(
        f"Batch did not complete within {max_wait_minutes} minutes: {batch_id}"
    )

Exponential Backoff for Production Systems

Fixed-interval polling works for simple scripts, but production systems should use exponential backoff to reduce unnecessary API calls and handle load gracefully.

import random
 
def wait_for_batch_with_backoff(
    client: anthropic.Anthropic,
    batch_id: str,
    initial_interval: float = 30.0,
    max_interval: float = 300.0,  # 5-minute maximum
    multiplier: float = 1.5,
    jitter: bool = True,
) -> anthropic.types.messages.MessageBatch:
    """
    Poll for batch completion using exponential backoff.
    Jitter prevents thundering herd issues when running multiple batches.
    """
    interval = initial_interval
 
    while True:
        batch = client.messages.batches.retrieve(batch_id)
 
        if batch.processing_status == "ended":
            return batch
 
        # Add jitter to spread out requests across multiple batch jobs
        wait_time = interval * (0.5 + random.random()) if jitter else interval
        print(f"Next poll in {wait_time:.1f}s...")
        time.sleep(wait_time)
 
        # Increase interval for next check, capped at max_interval
        interval = min(interval * multiplier, max_interval)

Step 3: Retrieving and Processing Results

Once the batch is complete, retrieve results and reconcile them with your source data.

Streaming Result Retrieval

def process_batch_results(
    client: anthropic.Anthropic,
    batch_id: str,
    original_documents: dict[str, dict],
) -> list[dict]:
    """
    Retrieve batch results and match them to source data.
 
    The `.results()` method streams results, making it memory-efficient
    even for very large batches.
    """
    results = []
 
    for result in client.messages.batches.results(batch_id):
        custom_id = result.custom_id
        original_doc = original_documents.get(custom_id, {})
 
        if result.result.type == "succeeded":
            message = result.result.message
            content_text = ""
            for block in message.content:
                if block.type == "text":
                    content_text = block.text
                    break
 
            results.append({
                "id": custom_id,
                "status": "success",
                "original": original_doc,
                "output": content_text,
                "input_tokens": message.usage.input_tokens,
                "output_tokens": message.usage.output_tokens,
            })
 
        elif result.result.type == "errored":
            # In the Python SDK, .error is the error *response*, so the
            # actual type lives one level deeper at .error.error.type
            err = result.result.error.error
            results.append({
                "id": custom_id,
                "status": "error",
                "original": original_doc,
                "error_type": err.type,      # e.g. invalid_request_error
                "error_message": err.message,
            })
 
        elif result.result.type == "expired":
            # Missed the 24-hour window. Not billed.
            results.append({
                "id": custom_id,
                "status": "expired",
                "original": original_doc,
            })
 
        elif result.result.type == "canceled":
            # A canceled batch still carries partial results. Drop this
            # branch and those requests vanish from your output silently.
            results.append({
                "id": custom_id,
                "status": "canceled",
                "original": original_doc,
            })
 
    return results
 
# Usage
documents_map = {doc["id"]: doc for doc in documents}
completed_batch = wait_for_batch_completion(client, batch.id)
results = process_batch_results(client, batch.id, documents_map)
 
# Save results
with open("batch_results.json", "w") as f:
    json.dump(results, f, indent=2)
 
success_count = sum(1 for r in results if r["status"] == "success")
error_count = sum(1 for r in results if r["status"] == "error")
print(f"Done: {success_count} succeeded, {error_count} errored")

Step 4: Production-Grade Batch Manager

The gap between a working script and a production system lies in error recovery, logging, and scalability. Here's a battle-tested batch manager class.

import logging
from dataclasses import dataclass, field
from datetime import datetime
import anthropic
 
logger = logging.getLogger(__name__)
 
@dataclass
class BatchJob:
    """Tracks state for a single batch job."""
    job_id: str
    batch_id: str | None = None
    status: str = "pending"
    submitted_at: datetime | None = None
    completed_at: datetime | None = None
    request_count: int = 0
    success_count: int = 0
    error_count: int = 0
    metadata: dict = field(default_factory=dict)
 
class BatchJobManager:
    """
    Production-ready batch job manager.
 
    Handles:
    - Automatic splitting of large request sets into multiple batches
    - Progress tracking and logging
    - Cost estimation from results
    """
 
    def __init__(
        self,
        client: anthropic.Anthropic,
        max_requests_per_batch: int = 5000,
    ):
        self.client = client
        self.max_requests_per_batch = max_requests_per_batch
        self.jobs: dict[str, BatchJob] = {}
 
    def submit_large_workload(
        self,
        all_requests: list[dict],
        job_prefix: str = "job",
    ) -> list[BatchJob]:
        """
        Split a large workload across multiple batches automatically.
        The hard cap is 100,000 requests, but the 256 MB size limit
        is what you actually hit first when a long system prompt is
        duplicated across every request. Split conservatively.
        """
        jobs = []
 
        for i in range(0, len(all_requests), self.max_requests_per_batch):
            chunk = all_requests[i:i + self.max_requests_per_batch]
            job_id = f"{job_prefix}_{i // self.max_requests_per_batch:04d}"
 
            try:
                logger.info(f"Submitting batch: {job_id} ({len(chunk)} requests)")
                batch = self.client.messages.batches.create(requests=chunk)
 
                job = BatchJob(
                    job_id=job_id,
                    batch_id=batch.id,
                    status="submitted",
                    submitted_at=datetime.now(),
                    request_count=len(chunk),
                )
                self.jobs[job_id] = job
                jobs.append(job)
                logger.info(f"Batch created: {batch.id}")
 
            except anthropic.APIError as e:
                logger.error(f"Failed to submit batch: {job_id} — {e}")
                job = BatchJob(
                    job_id=job_id,
                    status="failed",
                    metadata={"error": str(e)},
                )
                self.jobs[job_id] = job
                jobs.append(job)
 
        return jobs
 
    def estimate_cost(
        self,
        results: list[dict],
        model: str = "claude-sonnet-4-6",
    ) -> dict:
        """
        Calculate actual cost from batch results using Batches API pricing.
        """
        # Batches API pricing (50% discount applied)
        pricing = {
            "claude-sonnet-4-6": {"input": 1.50, "output": 7.50},
            "claude-sonnet-5": {"input": 1.00, "output": 5.00},
            "claude-haiku-4-5-20251001": {"input": 0.50, "output": 2.50},
            "claude-opus-5": {"input": 2.50, "output": 12.50},
        }
        model_pricing = pricing.get(model, pricing["claude-sonnet-4-6"])
 
        total_in = sum(r.get("input_tokens", 0) for r in results if r["status"] == "success")
        total_out = sum(r.get("output_tokens", 0) for r in results if r["status"] == "success")
 
        return {
            "total_input_tokens": total_in,
            "total_output_tokens": total_out,
            "input_cost_usd": round((total_in / 1_000_000) * model_pricing["input"], 4),
            "output_cost_usd": round((total_out / 1_000_000) * model_pricing["output"], 4),
            "total_cost_usd": round(
                (total_in / 1_000_000) * model_pricing["input"] +
                (total_out / 1_000_000) * model_pricing["output"],
                4
            ),
        }

Real-World Use Cases

Use Case 1: E-Commerce Product Description Generation

Automatically generate compelling product descriptions for an entire catalog.

def create_product_requests(products: list[dict]) -> list[dict]:
    """Build batch requests for product description generation."""
    requests = []
    for product in products:
        requests.append({
            "custom_id": f"product_{product['id']}",
            "params": {
                "model": "claude-haiku-4-5-20251001",  # Cost-effective for high volume
                "max_tokens": 300,
                "messages": [{
                    "role": "user",
                    "content": f"""Write a compelling 150-word product description for:
 
Product: {product['name']}
Category: {product['category']}
Key specs: {', '.join(product.get('specs', []))}
Price: ${product['price']}
 
Focus on benefits, not just features. End with a subtle call to action."""
                }]
            }
        })
    return requests

Using claude-haiku-4-5-20251001 with the Batches discount, you can generate descriptions for 100,000 products for a fraction of the cost of synchronous Sonnet calls.

Use Case 2: Bulk Sentiment Analysis

Process thousands of customer reviews for business intelligence.

def create_sentiment_requests(reviews: list[dict]) -> list[dict]:
    """Build batch requests for structured sentiment analysis."""
    requests = []
    for review in reviews:
        requests.append({
            "custom_id": f"review_{review['id']}",
            "params": {
                "model": "claude-haiku-4-5-20251001",
                "max_tokens": 150,
                "messages": [{
                    "role": "user",
                    "content": f"""Analyze this customer review and respond with JSON only.
 
Review: {review['text']}
 
Required format:
{{
  "sentiment": "positive|neutral|negative",
  "score": 1-5,
  "categories": ["quality", "price", "shipping", "support"],
  "summary": "Under 20 words",
  "requires_action": true|false
}}"""
                }]
            }
        })
    return requests

Use Case 3: Multilingual Content Pipeline

Translate articles into multiple languages in a single batch submission.

TARGET_LANGUAGES = [
    ("en", "English"), ("ja", "Japanese"),
    ("zh", "Simplified Chinese"), ("ko", "Korean"), ("es", "Spanish"),
]
 
def create_translation_requests(articles: list[dict], source_lang: str = "en") -> list[dict]:
    """Generate translation requests for all target languages."""
    requests = []
    for article in articles:
        for lang_code, lang_name in TARGET_LANGUAGES:
            if lang_code == source_lang:
                continue
            requests.append({
                "custom_id": f"article_{article['id']}_{lang_code}",
                "params": {
                    "model": "claude-sonnet-4-6",
                    "max_tokens": len(article["content"]) * 2,
                    "messages": [{
                        "role": "user",
                        "content": f"""Translate to {lang_name}. Maintain tone and formatting.
Return translated text only.
 
---
{article['content']}"""
                    }]
                }
            })
    return requests

Five languages × 200 articles = 1,000 requests submitted in one API call, processed asynchronously overnight.

Error Handling and Retry Logic

Types of Errors

• overloaded_error: Anthropic servers under heavy load. Retry later or reduce batch size.
• invalid_request_error: Malformed request (duplicate custom_id, missing required field, invalid parameter). Validate before submission.
• expired: Request wasn't processed within 24 hours. Resubmit the affected requests.

One piece of good news for retry design: errored, canceled, and expired results are all explicitly non-billable. I'd braced myself for paying twice on every retry loop; you only pay for what succeeded. A failure in one request also doesn't affect the rest of the batch.

Automatic Retry for Failed Requests

def retry_failed_requests(
    client: anthropic.Anthropic,
    original_requests: list[dict],
    results: list[dict],
    documents_map: dict[str, dict],
    max_retries: int = 3,
) -> list[dict]:
    """Automatically resubmit failed and expired requests."""
    failed_ids = {
        r["id"] for r in results
        if r["status"] in ("error", "expired")
    }
 
    if not failed_ids:
        return results
 
    retry_requests = [
        req for req in original_requests
        if req["custom_id"] in failed_ids
    ]
 
    logger.warning(f"Retrying {len(retry_requests)} failed requests (max {max_retries} attempts)")
    successful_results = [r for r in results if r["status"] == "success"]
 
    for attempt in range(max_retries):
        if not retry_requests:
            break
 
        batch = client.messages.batches.create(requests=retry_requests)
        wait_for_batch_completion(client, batch.id)
        # Passing an empty map here strands `original` on every retried
        # row, so the successes you recover can't be joined back
        retry_results = process_batch_results(client, batch.id, documents_map)
 
        successful_results.extend(r for r in retry_results if r["status"] == "success")
 
        still_failed = {r["id"] for r in retry_results if r["status"] in ("error", "expired")}
        retry_requests = [r for r in retry_requests if r["custom_id"] in still_failed]
        logger.info(f"Retry {attempt + 1}/{max_retries}: {len(retry_requests)} remaining")
 
    # Include permanently failed requests in final output
    for req in retry_requests:
        successful_results.append({
            "id": req["custom_id"],
            "status": "permanent_failure",
        })
 
    return successful_results

Cost Optimization Best Practices

Model Selection Strategy

Choosing the right model for each task type is as important as using the Batches API itself.

For simple classification, labeling, and short summaries, use claude-haiku-4-5-20251001. With the Batches discount, the per-token cost is extremely low while quality remains solid for straightforward tasks.

For detailed summarization, structured extraction, and translation, claude-sonnet-4-6 offers the best balance of quality and cost. This is the right default for most production workloads.

For complex reasoning, long-form generation, and expert-level analysis, invest in claude-opus-5 — $2.50 in, $12.50 out per MTok with the batch discount.

One caveat worth pausing on before you swap Sonnet 4.6 for Sonnet 5 on price alone: Claude 4.7 and later use a newer tokenizer that produces roughly 30% more tokens for the same text. A third off the unit price doesn't mean a third off the invoice for the same corpus. I'd rather run count_tokens against my own content than trust the rate card.

Combining Batches with Prompt Caching

For batches where all requests share a long system prompt, combine the Batches API with prompt caching (cache_control) for compounding savings:

request_with_cache = {
    "custom_id": "doc_001",
    "params": {
        "model": "claude-sonnet-4-6",
        "max_tokens": 500,
        "system": [
            {
                "type": "text",
                "text": "You are an expert financial analyst. Apply the following evaluation rubric...(long shared instructions)",
                # Default ephemeral TTL is 5 minutes. Batches routinely
                # take longer than that, so ask for the 1-hour cache.
                "cache_control": {"type": "ephemeral", "ttl": "1h"}
            }
        ],
        "messages": [
            {"role": "user", "content": "Analyze this document..."}
        ]
    }
}

Here's where I'd been too optimistic. A cache hit costs 0.1x the base input rate, a 5-minute write costs 1.25x, and a 1-hour write costs 2x. So "up to 90% off input" describes the portion you actually read from cache — net of the write premium, a 5-minute cache pays for itself after one read and a 1-hour cache after two. Batches run concurrently and asynchronously, so hits are best-effort; the docs cite observed hit rates anywhere from 30% to 98%.

The bigger constraint is that caching only touches the shared part of your input. If output dominates your bill, there simply isn't much for it to move. I've put the real breakdown from my pipeline further down, but the conclusion belongs here: the discounts apply to part of your input; the shape of your invoice is set by output tokens.

For a detailed guide on prompt caching implementation, see Claude API Cost Optimization Patterns.

Operational Gotchas That Aren't in the Official Docs

The Batches API docs cover the happy path well. The non-obvious behaviors only emerge after running production traffic for several weeks. Here are the ones that bit me, and how I work around them now.

processing_status Transitions Are Not Smooth Step-by-Step

The docs describe the state machine as in_progress → ended. In practice, batches get internally sharded — and shards can complete unevenly. On the translation stage of my pipeline, I see this two or three times per month: request_counts.succeeded jumps from zero to 4,800 out of 5,000 quickly, then the final 200 sit unprocessed for 40+ minutes before finishing.

The workaround is to make polling adaptive: once progress crosses 95%, shorten the poll interval so you don't miss the tail and risk timing out:

def adaptive_polling(client, batch_id, base_interval=60, fast_interval=15,
                     deadline_minutes=180):
    """Switch to a tighter poll interval once the batch is 95%+ done."""
    deadline = time.monotonic() + deadline_minutes * 60
    while time.monotonic() < deadline:
        batch = client.messages.batches.retrieve(batch_id)
        if batch.processing_status == "ended":
            return batch
 
        rc = batch.request_counts
        total = rc.processing + rc.succeeded + rc.errored + rc.expired
        done = rc.succeeded + rc.errored + rc.expired
        progress = done / total if total else 0
 
        interval = fast_interval if progress >= 0.95 else base_interval
        time.sleep(interval)
 
    # The API allows 24 hours; your downstream schedule usually can't
    raise TimeoutError(f"{deadline_minutes} minutes elapsed without ending: {batch_id}")

If a request stays in limbo long enough, it can drop into expired. Always have a retry queue ready for those.

custom_id Should Encode Both Business ID and Batch Session

The official rule is "max 64 characters of alphanumerics, hyphen, or underscore, unique within a single batch." Easy in isolation — but here's what trips real systems up: if today's nightly batch fails and you resubmit tomorrow with the same business IDs, your downstream result store ends up with overwritten records.

My convention: doc_<business-slug>__<YYYYMMDD>__<seq>. For example, doc_catalog_item_42__20260521__001. Two benefits at once: retrying the same business item on a different day creates a fresh record in the result store, and any audit log entry can be traced back to its exact submission day.

The 29-Day Clock Starts at created_at, Not at Completion

The docs mention 29-day retention as a one-line item, and this is the line I got wrong for a long time. Results are downloadable for 29 days from batch creation. The troubleshooting section says it plainly: check that it has been less than 29 days since created_at — "not processing ended_at time."

In practice that difference is a whole day of margin. A batch that took 23 hours to finish has 28 days and one hour left, not 29. If your archival job assumes "29 days after completion," only the requests near the boundary go missing — which is exactly the kind of failure nobody notices for a month. And the batch itself stays visible after expiry, still reporting processing_status: ended. The status tells you nothing about whether the data is still there.

I enforce a three-step pattern: (1) batch ends → results immediately serialized to R2 / KV; (2) a row-count integrity check the same day; (3) all of it triggered by cron, never by human memory. The deadline I write on the calendar is the day of created_at, not a date 29 days out.

SDK Bumps Move Types, Not Just Their Shape

What actually caught me wasn't the shape of a type but its address. Code written against the beta surface imports anthropic.types.beta.BetaMessageBatch, and that name simply doesn't exist once you move to client.messages.batches. processing_status is a literal type ("in_progress" | "ended" | "canceling"), so in a strict-mypy project the build is the only thing that fails, on a day when nobody touched the code.

Pin minor versions explicitly:

[project]
dependencies = [
  "anthropic>=0.40.0,<1.0.0",  # types move between namespaces on minor bumps
]

I review Anthropic SDK changelogs manually before merging dependabot PRs touching the Batches client. Keeping beta-header code (client.beta.messages.batches plus anthropic.types.beta.*) in separate modules from the standard path has also paid off.

Run a Preflight Check Before You Submit

Every gotcha above is the kind you discover after submitting. Anything that can be caught statically should be caught statically — nightly batches feel very different once you trust the submit path. These are the four checks I run immediately before create():

import json
import re
 
MAX_REQUESTS_PER_BATCH = 100_000      # official cap
MAX_BATCH_BYTES = 256 * 1024 * 1024   # 256 MB — the limit you hit first
CUSTOM_ID_RE = re.compile(r"^[a-zA-Z0-9_-]{1,64}$")
UNSUPPORTED_PARAMS = ("stream", "speed")  # rejected by batches
 
def preflight(requests: list[dict], budget_usd: float | None = None) -> None:
    """Static validation before submitting. Any failure aborts the submit."""
    if not requests:
        raise ValueError("Refusing to submit an empty batch (leaves an orphaned poller)")
    if len(requests) > MAX_REQUESTS_PER_BATCH:
        raise ValueError(f"Over per-batch limit: {len(requests)} requests")
 
    size = len(json.dumps(requests).encode("utf-8"))
    if size > MAX_BATCH_BYTES:
        raise ValueError(f"Over size limit: {size / 1024 / 1024:.1f} MB (413 on submit)")
 
    seen: set[str] = set()
    for r in requests:
        cid = r["custom_id"]
        if not CUSTOM_ID_RE.match(cid):
            raise ValueError(f"custom_id violates the spec: {cid}")
        if cid in seen:
            raise ValueError(f"Duplicate custom_id: {cid}")
        seen.add(cid)
 
        params = r["params"]
        for key in UNSUPPORTED_PARAMS:
            if key in params:
                raise ValueError(f"Parameter not supported in batches: {key} ({cid})")
        if params.get("max_tokens", 0) < 1:
            raise ValueError(f"max_tokens must be at least 1: {cid}")
 
    # Estimate the ceiling from max_tokens and stop runaway submissions
    if budget_usd is not None:
        est = sum(estimate_max_cost(r) for r in requests)
        if est > budget_usd:
            raise ValueError(f"Estimated cost exceeds budget: ${est:.2f} > ${budget_usd:.2f}")

Unglamorous, but mismatched result files went to zero once duplicate custom_ids were caught before submission. The budget_usd ceiling is insurance against the prompt-building side of the code: I once shipped a template bug that appended the full body text twice into every input, and this single check is what stopped it.

Five Ways Working Code Breaks the Moment You Batch It

Move a request that works synchronously into params and it tends to fail in one of five places. None of the errors mention batching, which is what makes them slow to diagnose.

The last row is the dangerous one. No error, no warning — just summaries attached to the wrong documents, one row out of alignment. I now compare the set of returned custom_ids against the set I submitted before anything gets written to storage.

Where the 50% Discount Doesn't Reach

It's easier to budget correctly if you memorize the exclusions rather than the inclusions.

• Fast mode: can't be combined with batches at all
• Claude Managed Agents sessions: stateful and interactive, so the Batch API discount doesn't apply
• Server-tool usage fees: web search is billed per 1,000 searches on top of tokens (the batch worker throttles web search per organization and retries what it throttles)

On the other side of the ledger: the 1M-token context window is priced at standard rates, and both the batch and caching discounts apply across the whole of it.

What Three Months of Invoices Actually Showed

"Up to 50% off" is the marketing number. Here is the same pipeline — the stages, volumes, and average token counts I was actually running from February to April 2026 — recomputed against today's published rates. The original version of this section carried arithmetic that didn't multiply out, plus per-token prices for a model that has since retired.

Pipeline Shape

• Unit of work: long-form generation → JA/EN translation → tag summarization → related-item extraction = 4 requests per unit
• Daily volume: 16 units/day, spread evenly across four output targets
• Monthly requests: 64/day × 30 days = roughly 1,920 requests/month before batching

Before/After Cost Breakdown

claude-sonnet-4-6 was the default; Japanese drafting used an Opus-class model.

Moving to Batches alone saves $45.05 (−50.0%). No surprise there — the unit price is simply halved.

The surprise was what came next. Put prompt caching on the ~1,500-token system prompt shared across all four stages, price the 1-hour writes at 2x and the reads at 0.1x, and assume a generous 90% hit rate. Across all four stages that saves $2.56 a month.

The monthly total lands at $42.49 — a 52.8% reduction versus sync. This article originally claimed 69.6%. I'd computed it as though caching discounted the entire input, forgetting that it only touches the shared prefix. Output tokens are about 80% of this pipeline's cost, and no amount of input-side cleverness moves that.

Before arguing about discounts, count what share of your bill is output tokens. It looks like a detour and it's the fastest route I've found.

Latency Distribution

Polling at a 60-second interval over 30 days, here's how long batches actually took to complete by size:

• 64-request batch: p50 = 4m 12s, p95 = 11m 38s, p99 = 19m 02s
• 320-request batch: p50 = 8m 47s, p95 = 22m 14s, p99 = 38m 51s
• 1,920-request batch: p50 = 18m 24s, p95 = 47m 09s, p99 = 1h 21m

Bigger batches reduce per-request overhead but stretch the p99 tail. I moved to flushing 16 units every six hours instead of bundling a full day at midnight, specifically so one unlucky p99 batch can't stall the entire next-morning pipeline.

How This Reshapes Scheduling

Once you split batches across time, your scheduler has to think in terms of "submit-time" and "harvest-time" separately. My nightly run looks like this:

• 02:00 — submit the long-form generation batch (webhook fires around 03:00 → harvest and submit the translation batch)
• 04:00 — harvest translations and submit the tag-summarization batch
• 06:00 — collect all results, persist them to durable storage, hand off to downstream delivery

A workload that's one synchronous job becomes three chained tasks. The tradeoff is worth it: lower bills, no rate-limit collisions with the user-facing app, and predictable nightly windows.

Route It to Batches, or Leave It Synchronous? Five Questions

Cost savings alone don't determine whether Batches is the right call. These are the five questions I ended up asking every time, after enough rounds of arguing with my own API bill.

1. Is the User Watching the Clock?

Sync APIs are mandatory when users are explicitly waiting — chat replies, code completions, real-time search. The Batches discount cannot rescue a feature that requires sub-second response.

The interesting middle ground is "soft real-time" work. For comment moderation I settled on a hybrid: the first time a user posts, moderation runs synchronously; subsequent edits flow into a 15-minute batch re-evaluation. Splits like this preserve the UX without paying premium pricing on every keystroke.

2. Do Later Requests Depend on Earlier Results?

Sequential reasoning chains (A's output feeds B's input) don't fit Batches well. Batches assume all requests in a submission are independent. If you need true sequencing, either submit two separate batches with a wait between them or stay synchronous. Forcing sequential work into a single batch is where I've watched teams burn weekends.

3. Does Monthly Volume Cross the Break-Even Line?

My rough heuristic: under 500 requests per month, sync is cheaper end-to-end. Batches introduces operational overhead — webhook receivers, result storage, timeout monitors, retry logic. The human-time cost of building and maintaining these systems wipes out the per-request savings at low volumes.

Past 5,000 monthly requests, the math reliably tips toward Batches. My 1,920/month sits right on the line, but I went with Batches anyway — I did not want to rebuild the pipeline the moment volume grew.

4. Are You Sitting at the Sync Rate Limit?

Anthropic enforces RPM and TPM limits per tier on the sync endpoint. Once you're pinned at the ceiling, every additional sync call risks pushing your user-facing traffic over the limit. Batches lives in a different priority queue with its own quota.

I learned this the hard way running bulk push-notification generation through the sync API one quarter — the rate-limit hits cascaded into real users getting errors in the chat feature. After that, my rule became: anything not directly tied to a user-facing response goes through Batches, no exceptions.

5. Can You Afford the Result-Persistence Tax?

The 29-day expiry isn't a sync-API concept. With Batches, the responsibility for retrieving, persisting, and verifying results lives entirely on the caller. Small teams without dedicated infrastructure engineers sometimes can't justify standing up the webhook + storage pipeline that's required to operate Batches safely.

Cloudflare Workers + R2 made this cheap for me; on AWS or GCP you should plan for at least a few hours of initial wiring plus ongoing maintenance.

My rule: if at least three of these five questions point toward Batches, route the workload through it. Otherwise stay synchronous.

Closing Thoughts and What to Try Next

The Claude API Messages Batches API is more than a 50% cost discount — designed for properly, it reshapes your scheduling model, decouples you from sync rate limits, and gives you headroom to grow. In my case the combined effect of Batches plus prompt caching was a 52.8% monthly reduction: 50 points from the unit price, 2.8 from caching.

What struck me most while revising this piece is that most of the errors were things that had been true when I published them. The beta call surface, the live model IDs, the round discount figure. Each was accurate on the day it went up, and every month I left it alone was another month of handing readers an outdated map.

Here's the one thing I'd ask you to check this week, if you're already running batches: put the 29-day deadline measured from created_at next to the timestamp when your results were actually archived. The gap between those two is how much sleep you're entitled to. Mine was shorter than I'd assumed, and finding that out was worth more than any of the pricing corrections above.

If you want a starting point, copy the BatchJobManager above and get just the polling and result-persistence paths running before you optimize anything else. For sync-API resilience patterns that complement this work, I've written more in Claude API Production Resilience Patterns, and Real-Time Chat with Streaming covers the other side of the architecture when you do need real-time. Reading the two together should give you a complete map of how to use the right tool for each part of your workload.

Thanks for reading. If it makes your next evening with a rate card a little shorter, that's enough.