⬡ API & SDK/2026-03-25Advanced

Building Edge AI Microservices with Claude API and Cloudflare Workers

Low-latency, highly scalable AI microservices built from Claude API and Cloudflare Workers — edge design patterns with production-ready code.

claude-api⁸³ cloudflare-workers⁵edge-computingmicroservicesproduction¹¹¹ serverless³

✦ Premium Article

Why Run AI at the Edge?

One of the biggest factors affecting user experience in AI applications is response time. Traditional serverless architectures concentrate requests in a single region, meaning users farther away from that region experience noticeably higher latency.

Cloudflare Workers run your code across 300+ edge locations worldwide, processing requests at the node closest to each user. Combine that with Claude API's powerful reasoning capabilities, and you've got the foundation for AI microservices that are both fast and scalable.

Below: the architecture, the patterns, and production-ready code for a Claude-powered AI microservice on Cloudflare Workers — from basic API calls through streaming, caching, rate limiting, and multi-endpoint routing.

What you'll learn:

Core architecture for calling Claude API from Cloudflare Workers
Streaming responses for real-time UI integration
Caching strategies using KV, D1, and R2 for cost optimization
Rate limiting, error handling, and retry patterns
An API gateway pattern for consolidating multiple AI endpoints

Who this is for: Intermediate-to-advanced engineers familiar with the Claude API basics who have some experience with Cloudflare Workers.

Core Architecture — Calling Claude API from Workers

Let's start with the simplest possible setup: a Cloudflare Worker that calls the Claude API directly.

Project Setup

# Create a new Worker project with Wrangler CLI
npm create cloudflare@latest claude-edge-service -- --type worker-typescript
cd claude-edge-service
 
# Install the Anthropic SDK
npm install @anthropic-ai/sdk
 
# Store your API key as a secret
npx wrangler secret put ANTHROPIC_API_KEY

Basic Worker Implementation

// src/index.ts
import Anthropic from "@anthropic-ai/sdk";
 
interface Env {
  ANTHROPIC_API_KEY: string;
}
 
export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    // Handle CORS preflight
    if (request.method === "OPTIONS") {
      return new Response(null, {
        headers: {
          "Access-Control-Allow-Origin": "*",
          "Access-Control-Allow-Methods": "POST, OPTIONS",
          "Access-Control-Allow-Headers": "Content-Type",
        },
      });
    }
 
    if (request.method !== "POST") {
      return new Response("Method not allowed", { status: 405 });
    }
 
    try {
      const { message, model } = await request.json<{
        message: string;
        model?: string;
      }>();
 
      const client = new Anthropic({ apiKey: env.ANTHROPIC_API_KEY });
 
      // Call the Claude API
      const response = await client.messages.create({
        model: model || "claude-sonnet-4-6",
        max_tokens: 1024,
        messages: [{ role: "user", content: message }],
      });
 
      const text =
        response.content[0].type === "text" ? response.content[0].text : "";
 
      return new Response(JSON.stringify({ response: text }), {
        headers: {
          "Content-Type": "application/json",
          "Access-Control-Allow-Origin": "*",
        },
      });
    } catch (error) {
      console.error("API Error:", error);
      return new Response(
        JSON.stringify({ error: "Internal server error" }),
        { status: 500, headers: { "Content-Type": "application/json" } }
      );
    }
  },
};

With just this basic pattern, you already have a globally distributed endpoint that can call the Claude API from anywhere in the world. But for production use, there's a lot more to consider. Let's layer in the essentials.

✦

Thank you for reading this far.

Continue Reading

What follows includes implementation code, benchmarks, and practical content we hope you'll find useful. This site runs without ads — server and development costs are supported entirely by members like you. If it's been helpful, we'd be truly grateful for your support.

WHAT YOU'LL LEARN

✦Edge computing and AI microservices architecture operating Claude API on Cloudflare Workers

✦Building serverless AI systems achieving global low-latency, pay-as-you-go, and auto-scaling

✦Hybrid AI infrastructure operation with edge inference execution and cloud integration

Secure payment via Stripe · Cancel anytime

✦

Unlock This Article

Get full access to the rest of this article. Buy once, read anytime. This site is ad-free — your support goes directly toward keeping it running.

Unlock all articles with Membership →

Implementing Streaming Responses

Claude API's streaming capability lets you send tokens to the client in real time as they're generated. Instead of waiting for the full response, users see text flowing in naturally — a much better experience for longer outputs.

// src/streaming.ts — Streaming endpoint with SSE
import Anthropic from "@anthropic-ai/sdk";
 
interface Env {
  ANTHROPIC_API_KEY: string;
}
 
async function handleStreamingRequest(
  request: Request,
  env: Env
): Promise<Response> {
  const { message, systemPrompt } = await request.json<{
    message: string;
    systemPrompt?: string;
  }>();
 
  const client = new Anthropic({ apiKey: env.ANTHROPIC_API_KEY });
 
  // Build an SSE stream using ReadableStream
  const stream = new ReadableStream({
    async start(controller) {
      const encoder = new TextEncoder();
 
      try {
        const response = client.messages.stream({
          model: "claude-sonnet-4-6",
          max_tokens: 2048,
          system: systemPrompt || "You are a helpful assistant.",
          messages: [{ role: "user", content: message }],
        });
 
        // Forward streaming events to the client
        response.on("text", (text) => {
          const data = JSON.stringify({ type: "text_delta", text });
          controller.enqueue(encoder.encode(`data: ${data}\n\n`));
        });
 
        // Send final usage stats when the stream completes
        const finalMessage = await response.finalMessage();
        const usage = {
          input_tokens: finalMessage.usage.input_tokens,
          output_tokens: finalMessage.usage.output_tokens,
        };
        const doneData = JSON.stringify({ type: "done", usage });
        controller.enqueue(encoder.encode(`data: ${doneData}\n\n`));
        controller.close();
      } catch (error) {
        const errData = JSON.stringify({
          type: "error",
          message: error instanceof Error ? error.message : "Unknown error",
        });
        controller.enqueue(encoder.encode(`data: ${errData}\n\n`));
        controller.close();
      }
    },
  });
 
  return new Response(stream, {
    headers: {
      "Content-Type": "text/event-stream",
      "Cache-Control": "no-cache",
      Connection: "keep-alive",
      "Access-Control-Allow-Origin": "*",
    },
  });
}
 
export { handleStreamingRequest };

Client-Side Consumption

// Frontend (Next.js / React)
async function streamChat(message: string) {
  const response = await fetch("https://your-worker.workers.dev/stream", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ message }),
  });
 
  const reader = response.body?.getReader();
  const decoder = new TextDecoder();
 
  while (reader) {
    const { done, value } = await reader.read();
    if (done) break;
 
    const chunk = decoder.decode(value);
    const lines = chunk.split("\n").filter((line) => line.startsWith("data: "));
 
    for (const line of lines) {
      const data = JSON.parse(line.slice(6));
      if (data.type === "text_delta") {
        // Append text to the chat UI
        appendToChat(data.text);
      } else if (data.type === "done") {
        // Log token usage
        console.log("Usage:", data.usage);
      }
    }
  }
}

Caching Strategies with KV, D1, and R2

Every Claude API call costs money, so caching identical responses is one of the most effective ways to cut costs while improving response times at the same time.

Response Caching with Cloudflare KV

// src/cache.ts — KV-based cache layer
interface Env {
  ANTHROPIC_API_KEY: string;
  AI_CACHE: KVNamespace; // Bound in wrangler.toml
}
 
// Generate a deterministic cache key from the request payload
async function generateCacheKey(
  model: string,
  messages: Array<{ role: string; content: string }>,
  systemPrompt?: string
): Promise<string> {
  const payload = JSON.stringify({ model, messages, systemPrompt });
  const hash = await crypto.subtle.digest(
    "SHA-256",
    new TextEncoder().encode(payload)
  );
  const hashArray = Array.from(new Uint8Array(hash));
  return hashArray.map((b) => b.toString(16).padStart(2, "0")).join("");
}
 
async function cachedCompletion(
  env: Env,
  model: string,
  messages: Array<{ role: string; content: string }>,
  systemPrompt?: string
): Promise<{ text: string; cached: boolean }> {
  const cacheKey = await generateCacheKey(model, messages, systemPrompt);
 
  // Check KV for a cached response
  const cached = await env.AI_CACHE.get(cacheKey);
  if (cached) {
    return { text: cached, cached: true };
  }
 
  // Cache miss — call the Claude API
  const client = new Anthropic({ apiKey: env.ANTHROPIC_API_KEY });
  const response = await client.messages.create({
    model,
    max_tokens: 2048,
    system: systemPrompt,
    messages,
  });
 
  const text =
    response.content[0].type === "text" ? response.content[0].text : "";
 
  // Store the response in KV with a 1-hour TTL
  await env.AI_CACHE.put(cacheKey, text, { expirationTtl: 3600 });
 
  return { text, cached: false };
}
 
export { cachedCompletion, generateCacheKey };

wrangler.toml Configuration

# wrangler.toml
name = "claude-edge-service"
main = "src/index.ts"
compatibility_date = "2026-03-01"
 
# KV bindings (caching)
[[kv_namespaces]]
binding = "AI_CACHE"
id = "your-kv-namespace-id"
 
# D1 binding (usage logs)
[[d1_databases]]
binding = "USAGE_DB"
database_name = "ai-usage-logs"
database_id = "your-d1-database-id"

Usage Tracking with D1

// src/usage-tracker.ts — Track API costs in D1
interface UsageRecord {
  model: string;
  input_tokens: number;
  output_tokens: number;
  cached: boolean;
  endpoint: string;
  timestamp: string;
}
 
async function trackUsage(db: D1Database, record: UsageRecord): Promise<void> {
  await db
    .prepare(
      `INSERT INTO usage_logs (model, input_tokens, output_tokens, cached, endpoint, timestamp)
     VALUES (?, ?, ?, ?, ?, ?)`
    )
    .bind(
      record.model,
      record.input_tokens,
      record.output_tokens,
      record.cached ? 1 : 0,
      record.endpoint,
      record.timestamp
    )
    .run();
}
 
// Get a weekly cost breakdown
async function getDailyCostReport(db: D1Database): Promise<Response> {
  const result = await db
    .prepare(
      `SELECT
        DATE(timestamp) as date,
        model,
        SUM(input_tokens) as total_input,
        SUM(output_tokens) as total_output,
        SUM(CASE WHEN cached = 1 THEN 1 ELSE 0 END) as cache_hits,
        COUNT(*) as total_requests
      FROM usage_logs
      WHERE timestamp >= datetime('now', '-7 days')
      GROUP BY DATE(timestamp), model
      ORDER BY date DESC`
    )
    .all();
 
  return new Response(JSON.stringify(result.results), {
    headers: { "Content-Type": "application/json" },
  });
}
 
export { trackUsage, getDailyCostReport };

Rate Limiting and Error Handling

In production, you need to protect against abuse and handle transient API failures gracefully.

Sliding Window Rate Limiter

// src/rate-limiter.ts — KV-based rate limiting
interface RateLimitConfig {
  maxRequests: number; // Max requests within the window
  windowSeconds: number; // Window duration in seconds
}
 
async function checkRateLimit(
  kv: KVNamespace,
  clientId: string,
  config: RateLimitConfig
): Promise<{ allowed: boolean; remaining: number; resetAt: number }> {
  const key = `ratelimit:${clientId}`;
  const now = Math.floor(Date.now() / 1000);
 
  // Get the current counter
  const data = await kv.get<{
    count: number;
    windowStart: number;
  }>(key, "json");
 
  if (!data || now - data.windowStart >= config.windowSeconds) {
    // Start a new window
    await kv.put(
      key,
      JSON.stringify({ count: 1, windowStart: now }),
      { expirationTtl: config.windowSeconds }
    );
    return {
      allowed: true,
      remaining: config.maxRequests - 1,
      resetAt: now + config.windowSeconds,
    };
  }
 
  if (data.count >= config.maxRequests) {
    return {
      allowed: false,
      remaining: 0,
      resetAt: data.windowStart + config.windowSeconds,
    };
  }
 
  // Increment the counter
  await kv.put(
    key,
    JSON.stringify({ count: data.count + 1, windowStart: data.windowStart }),
    { expirationTtl: config.windowSeconds }
  );
 
  return {
    allowed: true,
    remaining: config.maxRequests - data.count - 1,
    resetAt: data.windowStart + config.windowSeconds,
  };
}
 
export { checkRateLimit, RateLimitConfig };

Exponential Backoff Retry

// src/retry.ts — Retry with exponential backoff
async function retryWithBackoff<T>(
  fn: () => Promise<T>,
  maxRetries: number = 3,
  baseDelay: number = 1000
): Promise<T> {
  let lastError: Error | undefined;
 
  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    try {
      return await fn();
    } catch (error) {
      lastError = error instanceof Error ? error : new Error(String(error));
 
      // Don't retry non-retryable errors
      if (isNonRetryableError(lastError)) {
        throw lastError;
      }
 
      if (attempt < maxRetries) {
        // Exponential backoff with jitter
        const delay =
          baseDelay * Math.pow(2, attempt) + Math.random() * 1000;
        await new Promise((resolve) => setTimeout(resolve, delay));
      }
    }
  }
 
  throw lastError;
}
 
function isNonRetryableError(error: Error): boolean {
  const message = error.message.toLowerCase();
  return (
    message.includes("authentication") ||
    message.includes("invalid_api_key") ||
    message.includes("invalid_request")
  );
}
 
export { retryWithBackoff };

API Gateway Pattern — Unifying Multiple Endpoints

In real-world production systems, you'll typically use different prompts and models for different tasks. Using a Cloudflare Worker as a router, you can consolidate multiple AI endpoints into a single service.

// src/router.ts — AI microservice routing
import { cachedCompletion } from "./cache";
import { handleStreamingRequest } from "./streaming";
import { checkRateLimit } from "./rate-limiter";
import { trackUsage } from "./usage-tracker";
import { retryWithBackoff } from "./retry";
 
interface Env {
  ANTHROPIC_API_KEY: string;
  AI_CACHE: KVNamespace;
  RATE_LIMIT: KVNamespace;
  USAGE_DB: D1Database;
}
 
// Define your AI endpoints
const ENDPOINTS: Record<
  string,
  {
    model: string;
    systemPrompt: string;
    maxTokens: number;
  }
> = {
  "/api/chat": {
    model: "claude-sonnet-4-6",
    systemPrompt: "You are a helpful assistant.",
    maxTokens: 2048,
  },
  "/api/summarize": {
    model: "claude-haiku-4-5",
    systemPrompt:
      "Summarize the following text concisely. Output only the summary.",
    maxTokens: 512,
  },
  "/api/code-review": {
    model: "claude-sonnet-4-6",
    systemPrompt:
      "You are a senior code reviewer. Analyze the code and provide actionable feedback on bugs, performance, and best practices.",
    maxTokens: 4096,
  },
  "/api/translate": {
    model: "claude-haiku-4-5",
    systemPrompt:
      "Translate the following text to the target language. Output only the translation.",
    maxTokens: 2048,
  },
};
 
export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    const url = new URL(request.url);
    const path = url.pathname;
 
    // Health check
    if (path === "/health") {
      return new Response(JSON.stringify({ status: "ok" }), {
        headers: { "Content-Type": "application/json" },
      });
    }
 
    // Streaming endpoint
    if (path === "/api/stream") {
      return handleStreamingRequest(request, env);
    }
 
    // Standard AI endpoints
    const endpointConfig = ENDPOINTS[path];
    if (!endpointConfig) {
      return new Response("Not found", { status: 404 });
    }
 
    // Rate limit check
    const clientIp = request.headers.get("CF-Connecting-IP") || "unknown";
    const rateCheck = await checkRateLimit(env.RATE_LIMIT, clientIp, {
      maxRequests: 60,
      windowSeconds: 60,
    });
 
    if (!rateCheck.allowed) {
      return new Response(
        JSON.stringify({ error: "Rate limit exceeded" }),
        {
          status: 429,
          headers: {
            "Content-Type": "application/json",
            "X-RateLimit-Remaining": "0",
            "X-RateLimit-Reset": rateCheck.resetAt.toString(),
            "Retry-After": Math.ceil(
              rateCheck.resetAt - Date.now() / 1000
            ).toString(),
          },
        }
      );
    }
 
    try {
      const body = await request.json<{ message: string }>();
 
      // Cached Claude API call with retry
      const result = await retryWithBackoff(() =>
        cachedCompletion(
          env,
          endpointConfig.model,
          [{ role: "user", content: body.message }],
          endpointConfig.systemPrompt
        )
      );
 
      // Track usage asynchronously (don't block the response)
      // Expected output: one row inserted into the D1 usage_logs table
      void trackUsage(env.USAGE_DB, {
        model: endpointConfig.model,
        input_tokens: 0,
        output_tokens: 0,
        cached: result.cached,
        endpoint: path,
        timestamp: new Date().toISOString(),
      });
 
      return new Response(
        JSON.stringify({
          response: result.text,
          cached: result.cached,
        }),
        {
          headers: {
            "Content-Type": "application/json",
            "X-RateLimit-Remaining": rateCheck.remaining.toString(),
            "Access-Control-Allow-Origin": "*",
          },
        }
      );
    } catch (error) {
      return new Response(
        JSON.stringify({
          error: "Service temporarily unavailable",
        }),
        { status: 503, headers: { "Content-Type": "application/json" } }
      );
    }
  },
};

Deploying and Monitoring

Production wrangler.toml

# wrangler.toml — Production configuration
name = "claude-edge-service"
main = "src/router.ts"
compatibility_date = "2026-03-01"
 
# Production environment
[env.production]
name = "claude-edge-service-prod"
routes = [
  { pattern = "api.example.com/*", zone_name = "example.com" }
]
 
# KV (cache + rate limiting)
[[kv_namespaces]]
binding = "AI_CACHE"
id = "cache-namespace-id"
 
[[kv_namespaces]]
binding = "RATE_LIMIT"
id = "ratelimit-namespace-id"
 
# D1 (usage logs)
[[d1_databases]]
binding = "USAGE_DB"
database_name = "ai-usage-logs"
database_id = "database-id"

Deployment Commands

# Test locally in development
npx wrangler dev
 
# Deploy to production
npx wrangler deploy --env production
 
# Verify the deployment
# Expected output: {"response": "Hello! ...", "cached": false}
curl -X POST https://api.example.com/api/chat \
  -H "Content-Type: application/json" \
  -d '{"message": "Hello, Claude!"}'

Wrapping Up — Expanding the Possibilities of Edge AI

Combining the Claude API with Cloudflare Workers opens up a powerful new approach to deploying and operating AI applications. Here's a recap of the patterns we covered:

Basic calls: Direct Worker-to-Claude requests give you a global endpoint in minutes
Streaming: SSE-based real-time responses for a smoother user experience
Caching: KV-based response caching to cut costs and reduce latency simultaneously
Resilience: Rate limiting, retries, and error handling for production-grade stability
API gateway: A unified router that consolidates multiple AI endpoints into one service

When you're ready to try this in your own project, start with a single endpoint — something like /api/summarize — and expand from there. Cloudflare Workers' generous free tier (100,000 requests per day) makes it easy to experiment without upfront costs.

The convergence of edge computing and AI is only accelerating. Take the code from this guide as a starting point and build something uniquely yours.

Thank You for Reading

Claude Lab is ad-free, supported entirely by members like you. We publish practical guides daily with implementation code, benchmarks, and production-ready patterns. If you've found it useful, we'd love to have you on board.

✦Copy-paste ready implementation code
✦New advanced guides published daily
✦$5/mo or $10 for lifetime access

View Membership →