Building Production-Grade Multi-Agent Systems with Claude Code: Complete Implementation Guide

Why Multi-Agent Orchestration Matters

Modern enterprise automation demands sophisticated coordination beyond what a single agent can handle:

Financial Analysis — Retrieve market data from multiple sources in parallel, analyze in real-time, assess risk
Web Scraping at Scale — Deploy specialized scrapers for different site architectures, aggregate results intelligently
Complex Workflows — Multi-step processes from sales proposals to customer onboarding to contract generation
Fault Tolerance — Continue processing when individual agents fail, with automatic recovery mechanisms

Traditional approaches suffer from bloated state management and error-handling code. Claude Code combined with Agent SDK provides systematic solutions to these challenges.

Foundational Concept: Master-Worker Pattern

// Master Orchestrator Agent Implementation
const Anthropic = require("@anthropic-ai/sdk");
 
class AgentOrchestrator {
  constructor() {
    this.client = new Anthropic();
    this.agents = [];
  }
 
  // Register worker agents
  registerAgent(name, role, capabilities) {
    this.agents.push({ name, role, capabilities });
  }
 
  // Core task distribution engine
  async distributeTask(task, context) {
    // Analyze task requirements
    const analysis = await this.analyzeTask(task);
    
    // Select appropriate agents
    const assignedAgents = this.selectAgents(analysis, context);
    
    // Execute in parallel
    const results = await Promise.all(
      assignedAgents.map(agent => this.executeAgent(agent, task, context))
    );
 
    return this.aggregateResults(results);
  }
 
  // Task analysis for skill matching
  async analyzeTask(task) {
    const response = await this.client.messages.create({
      model: "claude-opus-4-6",
      max_tokens: 1024,
      messages: [
        {
          role: "user",
          content: `Analyze required skills for: "${task}". 
          Return JSON: {skills: [string], complexity: 1-10, estimatedTime: string}`
        }
      ]
    });
 
    const content = response.content[0];
    if (content.type === "text") {
      return JSON.parse(content.text);
    }
    return { skills: [], complexity: 5, estimatedTime: "unknown" };
  }
 
  // Select optimal agents
  selectAgents(analysis, context) {
    return this.agents.filter(agent => {
      const hasSkills = analysis.skills.some(skill => 
        agent.capabilities.includes(skill)
      );
      return hasSkills && (context.agentPreferences ? 
        context.agentPreferences.includes(agent.name) : true);
    });
  }
 
  // Execute on agent
  async executeAgent(agent, task, context) {
    try {
      const response = await this.client.messages.create({
        model: "claude-opus-4-6",
        max_tokens: 4096,
        messages: [
          {
            role: "user",
            content: `You are a ${agent.role} specializing in: ${agent.capabilities.join(", ")}.
            Task: ${task}
            Context: ${JSON.stringify(context)}
            
            Execute and provide detailed results with reasoning.`
          }
        ]
      });
 
      return {
        agentName: agent.name,
        result: response.content[0].type === "text" ? response.content[0].text : null,
        status: "success",
        tokensUsed: response.usage
      };
    } catch (error) {
      return {
        agentName: agent.name,
        error: error.message,
        status: "failed",
        tokensUsed: 0
      };
    }
  }
 
  // Aggregate results
  aggregateResults(results) {
    const successful = results.filter(r => r.status === "success");
    const failed = results.filter(r => r.status === "failed");
 
    return {
      successCount: successful.length,
      failureCount: failed.length,
      results: successful.map(r => ({
        agent: r.agentName,
        output: r.result
      })),
      errors: failed
    };
  }
}
 
// Usage
const orchestrator = new AgentOrchestrator();
orchestrator.registerAgent("data-analyst", "Financial Analyst", [
  "financial-analysis", "data-visualization", "trend-detection"
]);
orchestrator.registerAgent("web-expert", "Web Specialist", [
  "web-scraping", "html-parsing", "data-extraction"
]);
 
const result = await orchestrator.distributeTask(
  "Analyze market trends and extract competitor data",
  { agentPreferences: ["data-analyst", "web-expert"] }
);
 
console.log(result);

Pattern 1: Parallel Execution with Resilience

Use Case: Real-Time Multi-Source Data Fetching

Retrieve financial data (stocks, crypto, forex) from multiple sources in parallel. Survive individual source failures while maintaining overall workflow.

class ResilientDataFetcher {
  constructor(maxRetries = 3, timeoutMs = 30000) {
    this.client = new Anthropic();
    this.maxRetries = maxRetries;
    this.timeoutMs = timeoutMs;
  }
 
  // Per-source fetch with retry logic
  async fetchFromSource(source, query) {
    let lastError;
    for (let attempt = 1; attempt <= this.maxRetries; attempt++) {
      try {
        const response = await Promise.race([
          this.client.messages.create({
            model: "claude-opus-4-6",
            max_tokens: 2048,
            messages: [
              {
                role: "user",
                content: `Fetch from ${source}: "${query}". 
                Return: {source, timestamp, data, confidence}`
              }
            ]
          }),
          new Promise((_, reject) =>
            setTimeout(() => reject(new Error("Timeout")), this.timeoutMs)
          )
        ]);
 
        return JSON.parse(response.content[0].text);
      } catch (error) {
        lastError = error;
        const backoffMs = Math.pow(2, attempt) * 1000;
        console.log(`Attempt ${attempt} failed. Backing off ${backoffMs}ms...`);
        await new Promise(resolve => setTimeout(resolve, backoffMs));
      }
    }
 
    return {
      source,
      status: "failed",
      error: lastError.message,
      confidence: 0
    };
  }
 
  // Parallel fetching
  async fetchMultipleSources(sources, query) {
    const fetchTasks = sources.map(source => 
      this.fetchFromSource(source, query)
    );
 
    const results = await Promise.allSettled(fetchTasks);
 
    return {
      timestamp: new Date().toISOString(),
      query,
      results: results.map((result, index) => ({
        source: sources[index],
        status: result.status,
        data: result.status === "fulfilled" ? result.value : { error: result.reason }
      })),
      successRate: results.filter(r => r.status === "fulfilled").length / sources.length
    };
  }
}
 
// Usage
const fetcher = new ResilientDataFetcher(3, 30000);
const data = await fetcher.fetchMultipleSources(
  ["AlphaVantage", "CoinGecko", "OANDA"],
  "AAPL stock price"
);
 
console.log(`Success rate: ${(data.successRate * 100).toFixed(1)}%`);

Pattern 2: Dependency Graph Execution

Use Case: Automated Sales Proposal Generation

Multiple steps with explicit dependencies (customer fetch → needs analysis → proposal → quote). Execute in correct order, passing results forward.

class DependencyGraphExecutor {
  constructor() {
    this.client = new Anthropic();
    this.executionLog = {};
    this.resultCache = {};
  }
 
  // Execute workflow with dependency graph
  async executeWorkflow(graph, inputs) {
    const executionOrder = this.topologicalSort(graph);
 
    for (const taskId of executionOrder) {
      const task = graph[taskId];
 
      // Collect dependency results
      const dependencyResults = {};
      for (const depId of task.dependencies || []) {
        dependencyResults[depId] = this.resultCache[depId];
      }
 
      console.log(`Executing: ${taskId}`);
      const result = await this.executeTask(
        taskId,
        task,
        { ...inputs, dependencies: dependencyResults }
      );
 
      this.resultCache[taskId] = result;
      this.executionLog[taskId] = {
        timestamp: new Date().toISOString(),
        status: "completed",
        result
      };
    }
 
    return this.resultCache;
  }
 
  // Topological sorting
  topologicalSort(graph) {
    const visited = new Set();
    const stack = [];
 
    const visit = (node) => {
      if (visited.has(node)) return;
      visited.add(node);
 
      const task = graph[node];
      for (const dep of task.dependencies || []) {
        visit(dep);
      }
 
      stack.push(node);
    };
 
    for (const node of Object.keys(graph)) {
      visit(node);
    }
 
    return stack;
  }
 
  // Execute individual task
  async executeTask(taskId, task, context) {
    const response = await this.client.messages.create({
      model: "claude-opus-4-6",
      max_tokens: 4096,
      messages: [
        {
          role: "user",
          content: `
            Task: ${task.description}
            Context: ${JSON.stringify(context, null, 2)}
            
            Output format: ${task.outputFormat || "natural language"}
          `
        }
      ]
    });
 
    return response.content[0].type === "text" ? response.content[0].text : null;
  }
}
 
// Workflow definition
const proposalWorkflow = {
  "fetch_customer": {
    description: "Get customer profile from CRM",
    dependencies: [],
    outputFormat: "JSON"
  },
  "analyze_needs": {
    description: "Analyze customer needs",
    dependencies: ["fetch_customer"],
    outputFormat: "JSON"
  },
  "create_proposal": {
    description: "Generate proposal document",
    dependencies: ["fetch_customer", "analyze_needs"],
    outputFormat: "markdown"
  },
  "generate_quote": {
    description: "Create pricing quote",
    dependencies: ["analyze_needs"],
    outputFormat: "JSON"
  }
};
 
const executor = new DependencyGraphExecutor();
const results = await executor.executeWorkflow(proposalWorkflow, {
  customerId: "CUST-001"
});

Pattern 3: Load-Balanced Task Distribution

Use Case: Intelligent Web Scraping Assignment

Distribute scraping tasks to specialized agents based on site type. Balance load, monitor performance, reassign underperforming agents.

class LoadBalancedTaskDistributor {
  constructor() {
    this.client = new Anthropic();
    this.agents = [];
    this.taskQueue = [];
    this.metrics = {};
  }
 
  // Register agent with capacity
  registerAgent(id, specialization, maxConcurrentTasks = 3) {
    const agent = {
      id,
      specialization,
      maxConcurrentTasks,
      currentTasks: 0,
      completedTasks: 0,
      averageExecutionTime: 0,
      successRate: 1.0
    };
    this.agents.push(agent);
    this.metrics[id] = agent;
  }
 
  // Enqueue task
  enqueueTask(task) {
    this.taskQueue.push({
      id: Math.random().toString(36),
      ...task,
      createdAt: Date.now(),
      attempts: 0
    });
  }
 
  // Select agent using scoring
  selectBestAgent(task) {
    let bestAgent = null;
    let bestScore = -Infinity;
 
    for (const agent of this.agents) {
      if (agent.currentTasks >= agent.maxConcurrentTasks) continue;
 
      const loadScore = (agent.maxConcurrentTasks - agent.currentTasks) / agent.maxConcurrentTasks;
      const performanceScore = agent.successRate * (1 / (agent.averageExecutionTime || 1));
      
      let compatibilityScore = 0;
      if (task.requiredSkills) {
        const matches = task.requiredSkills.filter(skill =>
          agent.specialization.includes(skill)
        ).length;
        compatibilityScore = matches / task.requiredSkills.length;
      }
 
      const totalScore = (loadScore * 0.3) + (performanceScore * 0.4) + (compatibilityScore * 0.3);
 
      if (totalScore > bestScore) {
        bestScore = totalScore;
        bestAgent = agent;
      }
    }
 
    return bestAgent;
  }
 
  // Main processing loop
  async processTaskQueue() {
    while (this.taskQueue.length > 0) {
      const task = this.taskQueue.shift();
      const agent = this.selectBestAgent(task);
 
      if (!agent) {
        this.taskQueue.push(task);
        await new Promise(resolve => setTimeout(resolve, 1000));
        continue;
      }
 
      agent.currentTasks++;
      const startTime = Date.now();
 
      this.executeTaskOnAgent(agent, task)
        .then(result => {
          const executionTime = Date.now() - startTime;
 
          agent.completedTasks++;
          agent.averageExecutionTime = 
            (agent.averageExecutionTime * (agent.completedTasks - 1) + executionTime) / 
            agent.completedTasks;
          agent.successRate = 0.95 * agent.successRate + 0.05;
 
          console.log(`✓ Task ${task.id} by ${agent.id} in ${executionTime}ms`);
        })
        .catch(error => {
          task.attempts++;
          if (task.attempts < 3) {
            this.taskQueue.push(task);
          }
          agent.successRate = 0.95 * agent.successRate;
        })
        .finally(() => {
          agent.currentTasks--;
        });
 
      await new Promise(resolve => setTimeout(resolve, 100));
    }
  }
 
  // Execute on agent
  async executeTaskOnAgent(agent, task) {
    const response = await this.client.messages.create({
      model: "claude-opus-4-6",
      max_tokens: 2048,
      messages: [
        {
          role: "user",
          content: `Specializing in: ${agent.specialization.join(", ")}.
          
          Scrape task:
          URL: ${task.url}
          Target: ${task.targetData}
          
          Return as JSON.`
        }
      ]
    });
 
    return JSON.parse(response.content[0].text);
  }
}
 
// Usage
const distributor = new LoadBalancedTaskDistributor();
distributor.registerAgent("scraper-1", ["ecommerce", "json"], 2);
distributor.registerAgent("scraper-2", ["news", "article"], 3);
 
[
  { url: "shop.com", targetData: "prices", requiredSkills: ["ecommerce"] },
  { url: "news.com", targetData: "articles", requiredSkills: ["news"] }
].forEach(task => distributor.enqueueTask(task));
 
await distributor.processTaskQueue();

Token Optimization

Caching and Batch Processing

class TokenEfficientOrchestrator {
  constructor() {
    this.client = new Anthropic();
    this.responseCache = new Map();
  }
 
  // Cache identical queries
  async getCachedResponse(cacheKey, generator) {
    if (this.responseCache.has(cacheKey)) {
      return this.responseCache.get(cacheKey);
    }
 
    const response = await generator();
    this.responseCache.set(cacheKey, response);
    return response;
  }
 
  // Batch multiple tasks
  async batchExecute(tasks) {
    const batchPrompt = tasks.map((task, i) => 
      `Task ${i + 1}: ${task.description}`
    ).join("\n\n");
 
    const response = await this.client.messages.create({
      model: "claude-opus-4-6",
      max_tokens: 4096,
      messages: [
        {
          role: "user",
          content: `Process all tasks:\n\n${batchPrompt}`
        }
      ]
    });
 
    return response.content[0].text;
  }
}

Surviving Across Sessions — Externalizing Orchestrator State and Turning Failures into Learning

Every pattern so far assumes a single run. Parallelism, dependency graphs, load balancing — all of it lives between the moment the orchestrator starts and the moment it finishes.

Real pipelines rarely finish in one pass. You run them overnight and check the rest in the morning. You let the first wave of agents go, then run the rest only after reviewing results. The moment a run spans that gap, the model forgets everything from before. Start the orchestrator again and it has no idea which tasks are already done, so it redoes them from scratch.

Running the Dolice Labs article generation through agents as an indie developer, this is exactly where I got stuck: a nightly job stalled partway, and the next morning I spent longer reconstructing "how far did it get" from the logs than the run itself had taken.

Push orchestrator state out to files

The fix is plain. Write the orchestrator's progress out of the run and into files. Keep two: a human-readable summary and a machine-readable structured state.

from datetime import datetime
from pathlib import Path
import yaml
 
def save_orchestrator_state(run_dir: str, state: dict) -> None:
    """Write the orchestrator's progress out to two files."""
    run_path = Path(run_dir)
    run_path.mkdir(parents=True, exist_ok=True)
 
    # 1) A summary a human can scan at a glance
    summary = f"""# Run state (updated: {datetime.now():%Y-%m-%d %H:%M})
 
## Current phase
{state.get("current_phase", "unset")}
 
## Finished agents
{", ".join(state.get("done_agents", [])) or "none"}
 
## Pending tasks
{", ".join(state.get("pending_tasks", [])) or "none"}
 
## Key decisions
{state.get("decisions", "none")}
"""
    (run_path / "run_state.md").write_text(summary, encoding="utf-8")
 
    # 2) Structured state the machine reads on resume
    machine = {
        "updated_at": datetime.now().isoformat(),
        "current_phase": state.get("current_phase"),
        "done_agents": state.get("done_agents", []),
        "pending_tasks": state.get("pending_tasks", []),
    }
    with open(run_path / "run_state.yaml", "w", encoding="utf-8") as f:
        yaml.dump(machine, f, allow_unicode=True, default_flow_style=False)
 
 
def resume_orchestrator_state(run_dir: str) -> dict:
    """Load last run's state on resume; start empty if none exists."""
    state_file = Path(run_dir) / "run_state.yaml"
    if not state_file.exists():
        return {"current_phase": None, "done_agents": [], "pending_tasks": []}
    with open(state_file, encoding="utf-8") as f:
        return yaml.safe_load(f) or {}

Stack each finished agent into done_agents and keep only the next tasks in pending_tasks. On resume, read resume_orchestrator_state first and skip anything already done. This is where the earlier idea — tagging every agent with a unique ID in the logs — connects: logs give you visibility during a run, the state file gives you memory between runs.

Carry failures into the next run

Once state lives outside the run, you can go one step further: stop throwing away the corrections a human hands back, and stop letting the same mistake reappear next time.

import re
from collections import Counter
from datetime import datetime
from pathlib import Path
 
def collect_recurring_corrections(run_dir: str, threshold: int = 3) -> dict:
    """Pull only the corrections that recur at or above the threshold."""
    log_dir = Path(run_dir) / "_feedback"
    if not log_dir.exists():
        return {}
 
    corrections = []
    for log in log_dir.glob("*.md"):
        text = log.read_text(encoding="utf-8")
        corrections += re.findall(r"##\s*Correction\s*\n(.*?)(?=##|\Z)", text, re.DOTALL)
 
    normalized = [c.strip()[:100] for c in corrections]
    counts = Counter(normalized)
    return {k: v for k, v in counts.items() if v >= threshold}
 
 
def stage_improvements(run_dir: str, recurring: dict) -> None:
    """Record recurring corrections into pending_improvements.md as proposals.
    Never write them straight into the agent instructions; a human reviews first."""
    if not recurring:
        return
    pending = Path(run_dir) / "pending_improvements.md"
    stamp = datetime.now().strftime("%Y-%m-%d")
    lines = [f"- [{stamp}] x{count}: {issue[:80]}" for issue, count in recurring.items()]
    body = pending.read_text(encoding="utf-8") if pending.exists() else ""
    pending.write_text(body + f"\n\n## {stamp}\n" + "\n".join(lines), encoding="utf-8")
    print(f"Staged {len(recurring)} proposal(s). Review before applying.")

The deliberate pause is the point. A detected problem never gets written straight into the agent's instructions. It is staged as a proposal, reviewed by a human, and only then applied. An orchestrator quietly rewriting its own blueprint is the failure mode I fear most. Make it reversible before you automate it — that ordering is the one thing I never bend.

On the next run, the approved "known pitfalls" get prepended to each agent's prompt. No correction lands a third time. That alone visibly cut how often work bounced back to review.

Wrapping up

Claude Code + Agent SDK multi-agent orchestration solves:

Complex Logic — Parallel execution, dependencies, error handling
Scalability — Grows with agent count
Maintainability — Centralized orchestration
Cost Efficiency — Optimized token consumption

Master these patterns to build AI-driven enterprise workflows at scale.

Automating the Dolice Labs operations as an indie developer, I keep relearning that the more agents you add, the harder it gets to see who is waiting on whom. I skipped the orchestrator layer early on and paid for it; now I always funnel state sharing and error propagation through a single place. Complexity is something you contain by design, not by adding features.

See Claude Code Agent Guide and Agent SDK Intro for deeper learning.