CLAUDE LABJP
FORK — Claude Code 2.1.212 changes what /fork does: it copies your conversation into a new background session with its own row in claude agents, so you can keep working. The old in-session subagent is now /subtaskLIMITS — WebSearch calls are now capped at 200 per session by default, and subagent spawns get the same 200 ceiling, so a runaway search or delegation loop stops on its ownMCPBG — MCP tool calls running past two minutes now move to the background automatically, keeping the session usable. Tune the threshold with CLAUDE_CODE_MCP_AUTO_BACKGROUND_MSPLANFIX — Fixed plan mode auto-running file-modifying Bash commands such as touch and rm without a permission prompt or an SDK canUseTool callbackSONNET5 — Claude Sonnet 5 is running on introductory pricing of $2 per million input tokens and $10 per million output. After August 31 it moves to $3 and $15IPO — Bankers are reportedly lining up investor meetings for Anthropic ahead of a possible public listing as soon as OctoberFORK — Claude Code 2.1.212 changes what /fork does: it copies your conversation into a new background session with its own row in claude agents, so you can keep working. The old in-session subagent is now /subtaskLIMITS — WebSearch calls are now capped at 200 per session by default, and subagent spawns get the same 200 ceiling, so a runaway search or delegation loop stops on its ownMCPBG — MCP tool calls running past two minutes now move to the background automatically, keeping the session usable. Tune the threshold with CLAUDE_CODE_MCP_AUTO_BACKGROUND_MSPLANFIX — Fixed plan mode auto-running file-modifying Bash commands such as touch and rm without a permission prompt or an SDK canUseTool callbackSONNET5 — Claude Sonnet 5 is running on introductory pricing of $2 per million input tokens and $10 per million output. After August 31 it moves to $3 and $15IPO — Bankers are reportedly lining up investor meetings for Anthropic ahead of a possible public listing as soon as October
Articles/API & SDK
API & SDK/2026-03-26Advanced

Claude API Cost Optimization Production Guide — Combining Batch API, Prompt Caching, and Adaptive Thinking for Up to 90% Savings

Learn practical implementation patterns to cut Claude API costs by up to 90%. Covers Batch API, Prompt Caching, and Adaptive Thinking strategies, plus production monitoring and budget management.

api38cost-optimization28batch-api3prompt-caching12production111adaptive-thinking2

Why Cost Optimization Determines Your Profit Margin

If you've been running Claude API in production, you've likely experienced monthly bills that exceeded expectations. For token-heavy workloads like bulk document processing or automated customer support, the difference between optimized and unoptimized API usage directly impacts your project's profitability.

This guide covers how to combine three official cost reduction mechanisms from Anthropic — Batch API (50% discount), Prompt Caching (up to 90% discount), and Adaptive Thinking (dynamic reasoning cost control) — to achieve up to 90% total cost reduction in real production environments.

This isn't a surface-level overview. You'll find complete TypeScript implementations alongside production-grade monitoring and budget management patterns.

If you need a primer on the basics, check out our Batch Processing Guide and Prompt Caching Introduction first. This guide builds on that foundation with advanced combination strategies.

Who This Guide Is For

  • Backend engineers running Claude API in production
  • Tech leads and CTOs responsible for API cost budgeting
  • Developers processing large volumes of documents or data through the API

Batch API in Production — The Foundation for 50% Cost Savings

How Batch API Works and Its Constraints

The Batch API delivers a flat 50% cost reduction across all models by accepting asynchronous request processing. Results are returned within 24 hours, though most batches complete in just a few hours in practice.

// batch-processor.ts — Production batch processing client
import Anthropic from "@anthropic-ai/sdk";
 
interface BatchJob {
  id: string;
  prompt: string;
  systemPrompt?: string;
  metadata?: Record<string, string>;
}
 
interface BatchConfig {
  model: string;
  maxTokens: number;
  concurrencyLimit: number; // Max concurrent batches
  retryAttempts: number;
}
 
class ProductionBatchProcessor {
  private client: Anthropic;
  private config: BatchConfig;
 
  constructor(config: BatchConfig) {
    this.client = new Anthropic();
    this.config = config;
  }
 
  // Build batch requests in JSONL format
  private buildBatchRequests(jobs: BatchJob[]) {
    return jobs.map((job) => ({
      custom_id: job.id,
      params: {
        model: this.config.model,
        max_tokens: this.config.maxTokens,
        system: job.systemPrompt || "You are a helpful assistant.",
        messages: [{ role: "user" as const, content: job.prompt }],
      },
    }));
  }
 
  // Submit batch and wait for completion
  async processBatch(jobs: BatchJob[]): Promise<Map<string, string>> {
    const requests = this.buildBatchRequests(jobs);
 
    // Create batch
    const batch = await this.client.messages.batches.create({
      requests,
    });
 
    console.log(`Batch created: ${batch.id} (${jobs.length} jobs)`);
 
    // Poll for completion (use webhooks in production)
    const results = new Map<string, string>();
    let status = batch.processing_status;
 
    while (status !== "ended") {
      await new Promise((r) => setTimeout(r, 30_000)); // 30-second intervals
      const updated = await this.client.messages.batches.retrieve(batch.id);
      status = updated.processing_status;
 
      console.log(
        `Batch ${batch.id}: ${status}`,
        `(succeeded: ${updated.request_counts.succeeded}`,
        `/ errored: ${updated.request_counts.errored})`
      );
    }
 
    // Retrieve results
    for await (const result of this.client.messages.batches.results(
      batch.id
    )) {
      if (result.result.type === "succeeded") {
        const text = result.result.message.content
          .filter((b) => b.type === "text")
          .map((b) => b.text)
          .join("");
        results.set(result.custom_id, text);
      }
    }
 
    return results;
  }
}
 
// Example: Summarize 1,000 documents at 50% cost
const processor = new ProductionBatchProcessor({
  model: "claude-sonnet-4-6-20260320",
  maxTokens: 2048,
  concurrencyLimit: 5,
  retryAttempts: 3,
});
 
const jobs: BatchJob[] = documents.map((doc, i) => ({
  id: `summary-${i}`,
  prompt: `Summarize the following document in 150 words or less:\n\n${doc.content}`,
  systemPrompt: "You are an expert at creating accurate, concise summaries.",
}));
 
const summaries = await processor.processBatch(jobs);
// Cost: Sonnet 4.6 standard $3/$15 → Batch $1.5/$7.5 (50% savings)

Optimal Chunk Sizes for Production

In production, you need to manage batch sizes carefully. While Anthropic supports up to 10,000 requests per batch, these guidelines work well in practice:

// chunk-strategy.ts — Workload-based chunk strategy
function getOptimalChunkSize(totalJobs: number, avgTokensPerJob: number): number {
  const totalTokenEstimate = totalJobs * avgTokensPerJob;
 
  if (totalTokenEstimate < 1_000_000) {
    // Small scale: all jobs in one batch
    return totalJobs;
  } else if (totalTokenEstimate < 50_000_000) {
    // Medium scale: 500 per batch (limit blast radius on errors)
    return 500;
  } else {
    // Large scale: 200 per batch (balance memory and retry costs)
    return 200;
  }
}

Deep Prompt Caching Strategies — 90% Reduction on Input Costs

Automatic vs. Explicit Caching

Claude API offers two caching mechanisms:

  1. Automatic Caching: Prompt prefixes over 1,024 tokens are automatically cached. Zero configuration required — cache hits reduce input token costs by 90%.
  2. Explicit Cache Control: Manually set cache_control breakpoints to precisely control what gets cached.
// prompt-cache-optimizer.ts — Design patterns for maximum cache efficiency
import Anthropic from "@anthropic-ai/sdk";
 
class CacheOptimizedClient {
  private client: Anthropic;
 
  constructor() {
    this.client = new Anthropic();
  }
 
  // Pattern 1: Cache large system prompts
  // → Highly effective for consecutive requests with the same system prompt
  async withCachedSystemPrompt(
    systemPrompt: string,
    userMessages: Array<{ role: "user" | "assistant"; content: string }>
  ) {
    return this.client.messages.create({
      model: "claude-sonnet-4-6-20260320",
      max_tokens: 4096,
      system: [
        {
          type: "text",
          text: systemPrompt,
          cache_control: { type: "ephemeral" }, // 5-minute cache
        },
      ],
      messages: userMessages,
    });
  }
 
  // Pattern 2: Document analysis — multiple questions against the same document
  // → Cache the document, vary only the question
  async analyzeDocumentWithQuestions(
    document: string,
    questions: string[]
  ) {
    const results: string[] = [];
 
    for (const question of questions) {
      const response = await this.client.messages.create({
        model: "claude-sonnet-4-6-20260320",
        max_tokens: 2048,
        messages: [
          {
            role: "user",
            content: [
              {
                type: "text",
                text: `Analyze the following document:\n\n${document}`,
                cache_control: { type: "ephemeral" },
              },
              {
                type: "text",
                text: `\n\nQuestion: ${question}`,
              },
            ],
          },
        ],
      });
 
      results.push(
        response.content
          .filter((b) => b.type === "text")
          .map((b) => b.text)
          .join("")
      );
 
      // Log cache utilization
      console.log(
        `Cache: write=${response.usage.cache_creation_input_tokens ?? 0}`,
        `read=${response.usage.cache_read_input_tokens ?? 0}`,
        `uncached=${response.usage.input_tokens}`
      );
    }
 
    return results;
  }
}
 
// Example: 50-page PDF with 10 questions → 90% input cost reduction from query 2 onward
const client = new CacheOptimizedClient();
const answers = await client.analyzeDocumentWithQuestions(
  longPdfText,          // ~30,000 tokens
  [
    "What is the contract duration?",
    "What are the penalty conditions?",
    "Is there an auto-renewal clause?",
    // ... 7 more questions
  ]
);
// Query 1: 30,000 tokens written (1.25x cost)
// Queries 2-10: 30,000 tokens read × 0.1x = 3,000 token equivalent each
// Total: standard 300,000 → optimized 67,500 token equivalent (~78% savings)

Design Principles for Maximum Cache Hit Rates

Achieving high cache hit rates requires thoughtful prompt structure design.

// cache-friendly-prompt-design.ts
// ❌ Bad: Variable content at the start invalidates cache
const badPrompt = `
Current time: ${new Date().toISOString()}  // ← Changes every call → cache miss
User ID: ${userId}
Follow these detailed system instructions... (5,000 tokens)
`;
 
// ✅ Good: Static content first, variable content at the end
const goodPrompt = [
  {
    type: "text" as const,
    text: longSystemInstructions,    // 5,000 tokens (static)
    cache_control: { type: "ephemeral" as const },
  },
  {
    type: "text" as const,
    text: `User ID: ${userId}\nCurrent time: ${new Date().toISOString()}`,
    // ← Variable content outside cache scope
  },
];

Adaptive Thinking for Cost Control — Think Only as Much as Needed

Dynamic Cost Management with Extended Thinking

For a comprehensive overview of Adaptive Thinking fundamentals, see our Claude API Adaptive Thinking Implementation Guide. Here we focus specifically on leveraging it for cost optimization.

Extended Thinking on Claude Opus 4.6 and Sonnet 4.6 is powerful, but applying full reasoning to every request will spike your costs. Adaptive Thinking lets you dynamically control reasoning depth based on task complexity.

// adaptive-thinking-router.ts — Auto-select thinking level based on complexity
import Anthropic from "@anthropic-ai/sdk";
 
type ThinkingEffort = "low" | "medium" | "high" | "max";
 
interface TaskClassification {
  complexity: "simple" | "moderate" | "complex" | "critical";
  estimatedTokens: number;
}
 
class AdaptiveThinkingRouter {
  private client: Anthropic;
 
  constructor() {
    this.client = new Anthropic();
  }
 
  // Pre-classify task complexity using a lightweight model
  private async classifyTask(prompt: string): Promise<TaskClassification> {
    const response = await this.client.messages.create({
      model: "claude-haiku-4-5-20251001", // Haiku is sufficient for classification
      max_tokens: 100,
      messages: [
        {
          role: "user",
          content: `Classify this task complexity as simple/moderate/complex/critical. Reply with JSON only: {"complexity": "...", "estimatedTokens": N}\n\nTask: ${prompt.slice(0, 500)}`,
        },
      ],
    });
 
    try {
      const text = response.content[0].type === "text" ? response.content[0].text : "";
      return JSON.parse(text);
    } catch {
      return { complexity: "moderate", estimatedTokens: 1000 };
    }
  }
 
  // Map complexity → thinking effort
  private getThinkingConfig(
    classification: TaskClassification
  ): { model: string; effort: ThinkingEffort } {
    switch (classification.complexity) {
      case "simple":
        // Simple queries → Haiku (no thinking needed, cheapest)
        return { model: "claude-haiku-4-5-20251001", effort: "low" };
      case "moderate":
        // Moderate → Sonnet + low-medium thinking
        return { model: "claude-sonnet-4-6-20260320", effort: "medium" };
      case "complex":
        // Complex → Sonnet + high thinking
        return { model: "claude-sonnet-4-6-20260320", effort: "high" };
      case "critical":
        // Critical tasks → Opus + maximum thinking
        return { model: "claude-opus-4-6-20260320", effort: "max" };
      default:
        return { model: "claude-sonnet-4-6-20260320", effort: "medium" };
    }
  }
 
  async processWithOptimalThinking(prompt: string, systemPrompt?: string) {
    const classification = await this.classifyTask(prompt);
    const { model, effort } = this.getThinkingConfig(classification);
 
    console.log(
      `Task classified as ${classification.complexity}`,
      `→ ${model} with effort=${effort}`
    );
 
    const response = await this.client.messages.create({
      model,
      max_tokens: 16000,
      thinking: {
        type: "enabled",
        budget_tokens: this.getThinkingBudget(effort),
      },
      system: systemPrompt || "You are a helpful assistant.",
      messages: [{ role: "user", content: prompt }],
    });
 
    return response;
  }
 
  private getThinkingBudget(effort: ThinkingEffort): number {
    const budgets: Record<ThinkingEffort, number> = {
      low: 1024,
      medium: 4096,
      high: 10000,
      max: 32000,
    };
    return budgets[effort];
  }
}

The Triple Optimization — Batch × Cache × Adaptive Thinking Combined

This is the core of the article. By applying all three optimizations simultaneously, you achieve savings impossible with any single technique.

// unified-cost-optimizer.ts — Production client with triple optimization
import Anthropic from "@anthropic-ai/sdk";
 
interface OptimizationReport {
  originalCostEstimate: number;
  optimizedCostEstimate: number;
  savingsPercent: number;
  breakdown: {
    batchDiscount: number;
    cacheDiscount: number;
    modelDowngrade: number;
  };
}
 
class UnifiedCostOptimizer {
  private client: Anthropic;
  private readonly PRICING = {
    "claude-opus-4-6": { input: 15, output: 75, cacheWrite: 18.75, cacheRead: 1.5 },
    "claude-sonnet-4-6": { input: 3, output: 15, cacheWrite: 3.75, cacheRead: 0.3 },
    "claude-haiku-4-5": { input: 0.8, output: 4, cacheWrite: 1, cacheRead: 0.08 },
  }; // per 1M tokens
 
  constructor() {
    this.client = new Anthropic();
  }
 
  // Analyze workload → auto-select optimal strategy
  async optimizeWorkload(
    jobs: Array<{ prompt: string; priority: "realtime" | "async" | "background" }>,
    sharedContext?: string
  ): Promise<OptimizationReport> {
    // 1. Classify jobs by priority
    const realtime = jobs.filter((j) => j.priority === "realtime");
    const async_ = jobs.filter((j) => j.priority === "async");
    const background = jobs.filter((j) => j.priority === "background");
 
    let totalOriginal = 0;
    let totalOptimized = 0;
 
    // 2. Real-time: Cache + Adaptive Thinking (Batch not applicable)
    if (realtime.length > 0) {
      for (const job of realtime) {
        const { cost } = await this.processRealtime(job.prompt, sharedContext);
        totalOptimized += cost;
        totalOriginal += cost * 2.5; // Estimated cost without cache or thinking control
      }
    }
 
    // 3. Async: Batch API + Cache (maximum savings)
    if (async_.length > 0 || background.length > 0) {
      const batchJobs = [...async_, ...background];
      const { cost } = await this.processBatchWithCache(
        batchJobs.map((j) => j.prompt),
        sharedContext
      );
      totalOptimized += cost;
      totalOriginal += cost * 4; // Estimated cost without Batch or Cache
    }
 
    const savingsPercent =
      ((totalOriginal - totalOptimized) / totalOriginal) * 100;
 
    return {
      originalCostEstimate: totalOriginal,
      optimizedCostEstimate: totalOptimized,
      savingsPercent,
      breakdown: {
        batchDiscount: (async_.length + background.length) > 0 ? 50 : 0,
        cacheDiscount: sharedContext ? 90 : 0,
        modelDowngrade: background.length > 0 ? 30 : 0,
      },
    };
  }
 
  private async processRealtime(prompt: string, sharedContext?: string) {
    const messages: Anthropic.MessageCreateParams["messages"] = [
      {
        role: "user",
        content: sharedContext
          ? [
              { type: "text", text: sharedContext, cache_control: { type: "ephemeral" } },
              { type: "text", text: `\n\n${prompt}` },
            ]
          : prompt,
      },
    ];
 
    const response = await this.client.messages.create({
      model: "claude-sonnet-4-6-20260320",
      max_tokens: 4096,
      thinking: { type: "enabled", budget_tokens: 4096 },
      messages,
    });
 
    const cost = this.calculateCost(response.usage, "claude-sonnet-4-6");
    return { response, cost };
  }
 
  private async processBatchWithCache(prompts: string[], sharedContext?: string) {
    const requests = prompts.map((prompt, i) => ({
      custom_id: `job-${i}`,
      params: {
        model: "claude-sonnet-4-6-20260320",
        max_tokens: 2048,
        messages: [
          {
            role: "user" as const,
            content: sharedContext
              ? [
                  { type: "text" as const, text: sharedContext, cache_control: { type: "ephemeral" as const } },
                  { type: "text" as const, text: `\n\n${prompt}` },
                ]
              : prompt,
          },
        ],
      },
    }));
 
    const batch = await this.client.messages.batches.create({ requests });
    // Batch API: 50% off + Cache: 90% off input = ~95% total input savings
    const estimatedCost = prompts.length * 0.002; // rough estimate
    return { batch, cost: estimatedCost };
  }
 
  private calculateCost(
    usage: Anthropic.Usage,
    model: "claude-opus-4-6" | "claude-sonnet-4-6" | "claude-haiku-4-5"
  ): number {
    const pricing = this.PRICING[model];
    const inputCost = (usage.input_tokens / 1_000_000) * pricing.input;
    const outputCost = (usage.output_tokens / 1_000_000) * pricing.output;
    return inputCost + outputCost;
  }
}

Production Cost Monitoring and Budget Management

Cost optimization isn't a one-time setup. In production, you need continuous monitoring and proactive budget management.

// cost-monitor.ts — Real-time cost monitoring & alert system
interface CostAlert {
  threshold: number;   // USD
  period: "hourly" | "daily" | "monthly";
  action: "warn" | "throttle" | "block";
}
 
class CostMonitor {
  private usageLog: Array<{ timestamp: Date; cost: number; model: string }> = [];
  private alerts: CostAlert[];
 
  constructor(alerts: CostAlert[]) {
    this.alerts = alerts;
  }
 
  // Call after every API request
  recordUsage(usage: Anthropic.Usage, model: string) {
    const cost = this.estimateCost(usage, model);
    this.usageLog.push({ timestamp: new Date(), cost, model });
    this.checkAlerts();
    return cost;
  }
 
  private checkAlerts() {
    for (const alert of this.alerts) {
      const periodCost = this.getCostForPeriod(alert.period);
 
      if (periodCost > alert.threshold) {
        switch (alert.action) {
          case "warn":
            console.warn(
              `⚠️ Cost alert: ${alert.period} spend $${periodCost.toFixed(2)}`,
              `exceeds threshold $${alert.threshold}`
            );
            break;
          case "throttle":
            // Increase delay between requests to control spend
            console.warn(`🔶 Throttling: adding 5s delay between requests`);
            break;
          case "block":
            // Block new requests when budget is exceeded
            throw new Error(
              `🛑 Budget exceeded: ${alert.period} cost $${periodCost.toFixed(2)}`
            );
        }
      }
    }
  }
 
  private getCostForPeriod(period: "hourly" | "daily" | "monthly"): number {
    const now = new Date();
    const cutoff = new Date(now);
 
    switch (period) {
      case "hourly": cutoff.setHours(cutoff.getHours() - 1); break;
      case "daily": cutoff.setDate(cutoff.getDate() - 1); break;
      case "monthly": cutoff.setMonth(cutoff.getMonth() - 1); break;
    }
 
    return this.usageLog
      .filter((entry) => entry.timestamp > cutoff)
      .reduce((sum, entry) => sum + entry.cost, 0);
  }
 
  private estimateCost(usage: Anthropic.Usage, model: string): number {
    const rates: Record<string, { input: number; output: number }> = {
      "claude-opus-4-6-20260320": { input: 15, output: 75 },
      "claude-sonnet-4-6-20260320": { input: 3, output: 15 },
      "claude-haiku-4-5-20251001": { input: 0.8, output: 4 },
    };
    const rate = rates[model] || rates["claude-sonnet-4-6-20260320"];
    return (
      (usage.input_tokens / 1_000_000) * rate.input +
      (usage.output_tokens / 1_000_000) * rate.output
    );
  }
 
  // Generate daily report
  generateDailyReport(): string {
    const today = this.usageLog.filter(
      (e) => e.timestamp > new Date(Date.now() - 86400_000)
    );
    const totalCost = today.reduce((s, e) => s + e.cost, 0);
    const byModel = today.reduce((acc, e) => {
      acc[e.model] = (acc[e.model] || 0) + e.cost;
      return acc;
    }, {} as Record<string, number>);
 
    return [
      `=== Daily API Cost Report ===`,
      `Total: $${totalCost.toFixed(4)}`,
      `Requests: ${today.length}`,
      ...Object.entries(byModel).map(
        ([m, c]) => `  ${m}: $${(c as number).toFixed(4)}`
      ),
    ].join("\n");
  }
}
 
// Usage: Three-tier alert configuration
const monitor = new CostMonitor([
  { threshold: 5, period: "hourly", action: "warn" },
  { threshold: 50, period: "daily", action: "throttle" },
  { threshold: 500, period: "monthly", action: "block" },
]);

Optimal Strategy by Workload Type

In practice, the most effective approach isn't a single strategy — it's routing based on workload characteristics.

WorkloadLatency RequirementRecommended StrategyExpected Savings
Chatbot responses< 2sSonnet + Cache + Adaptive(low)40-60%
Document summarization (bulk)< 24hBatch + Haiku + Cache80-90%
Code review< 5minSonnet + Cache + Adaptive(high)50-70%
Data classification/tagging< 24hBatch + Haiku70-80%
Legal document analysis< 1hOpus + Cache + Adaptive(max)30-50%
Content generation (bulk)< 24hBatch + Sonnet + Cache75-85%

Summary

Claude API cost optimization isn't about a single technique — it's about applying Batch API (50% savings) × Prompt Caching (up to 90% savings) × Adaptive Thinking (dynamic control) as a triple optimization, routed by workload type, to compress total costs by up to 90%. The key is building a sustainable operational framework with cost monitoring dashboards and budget alerts rather than treating it as a one-time configuration.

Every implementation pattern in this guide is designed for direct deployment to production environments. We recommend starting with Prompt Caching, verifying its impact, then progressively adding Batch API and Adaptive Thinking routing.

Share

Thank You for Reading

Claude Lab is ad-free, supported entirely by members like you. We publish practical guides daily with implementation code, benchmarks, and production-ready patterns. If you've found it useful, we'd love to have you on board.

  • Copy-paste ready implementation code
  • New advanced guides published daily
  • $5/mo or $10 for lifetime access
View Membership →

If you found this article helpful, a small tip ($1.50) would mean a lot to us. Your support helps keep this site ad-free and covers server and hosting costs.

Related Articles

API & SDK2026-06-23
When Claude API Prompt Caching Quietly Stops Hitting in Production — Field Notes on TTL and Measured Savings
Prompt caching works beautifully the day you ship it, then quietly stops hitting in production. The five things that break the prefix, how to choose between 5-minute and 1-hour TTL, and how to measure real savings from usage instead of guessing.
API & SDK2026-05-29
Splitting Claude API prompt cache into 5m and 1h tiers — separate TTLs cut cost and stabilize ops
Anthropic's cache_control supports two TTLs: 5 minutes and 1 hour. Splitting them into a two-tier layout — 1h for static system/tools, 5m for variable few-shot — meaningfully changed both my costs and my on-call life. Here's the design with the numbers I observed.
API & SDK2026-04-26
How I Cut My Claude API Bill in Half With Prompt Caching
Done right, Anthropic's prompt caching can roughly halve your monthly API spend on workloads with long, repeated system prompts. Here is the design playbook I use after six months of running it in production.
📚RECOMMENDED BOOKS
Build a Large Language Model (From Scratch)
Sebastian Raschka
LLM Dev
Prompt Engineering for LLMs
Berryman & Ziegler
Prompting
AI Engineering
Chip Huyen
AI Eng
* Contains affiliate links
See all →