Combining Haiku 4.5, Streaming, and Prompt Caching to Cut Costs in a Personal App — An Implementation Record

About six months after integrating Claude API into my personal app, something felt off. The feature was working, but at the end of each month, the API bill was climbing faster than the user count.

I sat down with the request logs and traced the problem to three overlapping issues: the model was heavier than the task required; streaming was introducing instability on certain screens rather than improving the experience; and a nearly-static 800-token system prompt was being re-sent with every single request.

I tried to fix all three individually first, and failed at each attempt. Prompt caching in particular — I put the breakpoint in the wrong place and nearly concluded it "didn't work." It wasn't just failing to save money. It was actively costing more, because cache writes are billed at a premium.

This is the record of getting those three techniques to work together, mistakes included. I've also gone back and recalculated, from published pricing, something I originally judged by feel: which lever actually moved the bill, and by how much.

Why Three Techniques Were Needed Together

Before combining them, I tried each approach individually. That's where the real learning happened.

Haiku 4.5 alone delivered on the promise of being faster and cheaper than Sonnet. But switching models without updating prompts introduced inconsistency. Instructions that Sonnet followed reliably were occasionally ignored by Haiku — not randomly, but in a pattern I had to dig into.

Streaming alone improved the perceived responsiveness of the app. But it introduced new failure modes: dropped connections, partial response rendering, and edge cases in the UI that hadn't existed with the simple request-response model. Streaming looks simple on the surface but has hidden complexity in implementation.

Prompt caching alone didn't hit the theoretical savings I expected. The reason, as I later discovered, was a misunderstanding of where to place the cache breakpoint. Placing it incorrectly doesn't just fail to save money — it adds cache-write costs on top.

Combining all three required understanding how they interact. Here's the full sequence.

Environment and App Profile

The app is a personalized content app for iOS and Android, built and operated by me as an indie developer — which matters here, because it means there's no separate infrastructure budget to absorb a bad month. Users enter text, Claude generates a response, and the result is displayed in-app. The request profile looks like this:

• System prompt: ~800 tokens (app-specific instructions, mostly static)
• User input: 50–200 tokens (changes every request)
• Expected output: 100–400 tokens

Backend: Node.js + TypeScript. Claude integration: the official Anthropic SDK. Monthly active requests: tens of thousands.

import Anthropic from "@anthropic-ai/sdk";
 
const client = new Anthropic({
  apiKey: process.env.ANTHROPIC_API_KEY,
});

Step 1: Migrating to Haiku 4.5 and Prompt Redesign

What Went Wrong with the Initial Migration

Haiku 4.5 occasionally failed to follow instructions that Sonnet handled without issue. After analyzing request logs, I identified the pattern: instructions placed in the second half of a long system prompt were being deprioritized.

The fix was to restructure the system prompt, moving the most critical instructions to the top and reducing the total density. I also found that Haiku responds better to instructions written in flowing sentences rather than bullet-point lists.

// Before: overpacked system prompt with mixed priorities
const systemPromptV1 = `
You are the assistant for [App Name].
- Respond to user input
- Always respond in Japanese
- Use a friendly tone
- Keep responses under 200 characters
- Avoid technical jargon
- Include 1-2 emoji
- Avoid negative statements
- Include encouraging content
... (continues)
`;
 
// After: critical instructions consolidated at the top
const systemPromptV2 = `
You are the assistant for [App Name].
Respond to user input in Japanese, within 200 characters, in an upbeat and encouraging tone.
 
Required:
- Always respond in Japanese
- Keep responses to 100-150 characters
- Include 1-2 emoji
- Maintain a positive tone
`;

Instruction compliance stabilized noticeably after this change.

Model Configuration

const response = await client.messages.create({
  model: "claude-haiku-4-5-20251001",
  max_tokens: 512,
  system: systemPromptV2,
  messages: [
    {
      role: "user",
      content: userInput,
    },
  ],
});

Step 2: Streaming Implementation and Its Hidden Issues

When Streaming Helps vs. When It Doesn't

Streaming improves experience when output is long and users are consciously waiting. For short output — under 100 characters — streaming can actually feel worse. Tokens appearing one by one look more natural when the output is substantial; for brief responses, it looks choppy.

Since most of my app's responses are under 200 characters, I added conditional routing:

type ContentRequest = {
  userInput: string;
  expectedLength: "short" | "long";
};
 
async function generateContent(req: ContentRequest): Promise<string> {
  if (req.expectedLength === "short") {
    return await generateWithoutStreaming(req.userInput);
  } else {
    return await generateWithStreaming(req.userInput);
  }
}

The Core Streaming Pattern

The most important part of streaming in a production mobile app isn't the happy path — it's handling disconnections and partial responses.

async function generateWithStreaming(userInput: string): Promise<string> {
  const chunks: string[] = [];
 
  try {
    const stream = await client.messages.stream({
      model: "claude-haiku-4-5-20251001",
      max_tokens: 1024,
      system: systemPromptV2,
      messages: [{ role: "user", content: userInput }],
    });
 
    for await (const chunk of stream) {
      if (
        chunk.type === "content_block_delta" &&
        chunk.delta.type === "text_delta"
      ) {
        chunks.push(chunk.delta.text);
        // Forward to UI via WebSocket/SSE here if needed
      }
    }
 
    const finalMessage = await stream.finalMessage();
 
    if (finalMessage.stop_reason !== "end_turn") {
      console.warn(
        `Unexpected stop_reason: ${finalMessage.stop_reason}`,
        { input: userInput }
      );
    }
 
    return chunks.join("");
  } catch (error) {
    if (error instanceof Anthropic.APIConnectionError) {
      console.error("Stream connection error:", error.message);
      throw error;
    }
    throw error;
  }
}

The stop_reason Problem I Didn't Expect

After deployment, logs showed stop_reason: "max_tokens" appearing sporadically. I had set max_tokens: 512, which should have been more than enough for 200-character Japanese responses.

The issue: Japanese characters can span multiple tokens. A "200-character" constraint in the system prompt doesn't map cleanly to a token budget. Responses of 400–500 characters were occasionally being generated, running into the token limit mid-output.

The fix was to reduce max_tokens to 256 and tighten the prompt instruction from "under 200 characters" to "approximately 100–150 characters." The buffer matters.

Step 3: Prompt Caching Design

Understanding the Caching Boundary Correctly

Three rules to internalize:

1. A cache entry is created when a request includes cache_control: { type: "ephemeral" } for the first time
2. The cache hits when all content before the cache breakpoint is identical in a subsequent request
3. Cache entries expire after 5 minutes (ephemeral)

My initial mistake was placing the cache breakpoint after the user input — the part that changes every request. This meant the cache never matched.

// Wrong: breakpoint placed after user input (which changes every request)
const badRequest = {
  model: "claude-haiku-4-5-20251001",
  max_tokens: 256,
  system: [{ type: "text", text: systemPromptV2 }],
  messages: [
    {
      role: "user",
      content: [
        {
          type: "text",
          text: userInput,
          cache_control: { type: "ephemeral" }, // ← breakpoint after variable content
        },
      ],
    },
  ],
};
 
// Correct: breakpoint placed on the fixed system prompt
const goodRequest = {
  model: "claude-haiku-4-5-20251001",
  max_tokens: 256,
  system: [
    {
      type: "text",
      text: systemPromptV2, // ← this is static
      cache_control: { type: "ephemeral" }, // ← breakpoint on fixed content
    },
  ],
  messages: [
    {
      role: "user",
      content: userInput, // ← this changes each request, which is fine
    },
  ],
};

Monitoring Cache Hit Rate

The API response includes token breakdown fields: cache_creation_input_tokens and cache_read_input_tokens. Tracking these tells you whether the cache is actually working.

interface UsageStats {
  inputTokens: number;
  outputTokens: number;
  cacheCreationTokens: number;
  cacheReadTokens: number;
}
 
function extractUsage(response: Anthropic.Message): UsageStats {
  return {
    inputTokens: response.usage.input_tokens,
    outputTokens: response.usage.output_tokens,
    cacheCreationTokens: response.usage.cache_creation_input_tokens ?? 0,
    cacheReadTokens: response.usage.cache_read_input_tokens ?? 0,
  };
}
 
function calculateCacheHitRate(stats: UsageStats[]): number {
  const cacheHits = stats.filter((s) => s.cacheReadTokens > 0).length;
  return (cacheHits / stats.length) * 100;
}

My initial cache hit rate was 30–40%. The 5-minute TTL meant that during quiet periods, the cache expired before the next request arrived.

Keeping the Cache Warm

The solution was to proactively refresh the cache before it expired:

async function warmUpCache(): Promise<void> {
  try {
    await client.messages.create({
      model: "claude-haiku-4-5-20251001",
      max_tokens: 1, // minimal tokens to keep cost low
      system: [
        {
          type: "text",
          text: systemPromptV2,
          cache_control: { type: "ephemeral" },
        },
      ],
      messages: [{ role: "user", content: "ping" }],
    });
    console.log("Cache warmed up at:", new Date().toISOString());
  } catch (error) {
    // Warmup failure is non-critical
    console.warn("Cache warmup failed:", error);
  }
}
 
// Refresh every 4 minutes (before the 5-minute TTL expires)
setInterval(warmUpCache, 4 * 60 * 1000);

After adding this, cache hit rate stabilized at 70–80%.

The warmup calls themselves incur cache-write charges, though — something I wasn't tracking at the time. Whether this setup actually came out ahead is a question I revisit with numbers later in this article.

Step 4: The Combined Implementation

Unified Request Function

interface GenerateOptions {
  userInput: string;
  streaming: boolean;
  onStreamChunk?: (chunk: string) => void;
}
 
async function generateOptimized(
  options: GenerateOptions
): Promise<{ text: string; usage: UsageStats }> {
  const { userInput, streaming, onStreamChunk } = options;
 
  const systemConfig = [
    {
      type: "text" as const,
      text: systemPromptV2,
      cache_control: { type: "ephemeral" } as const,
    },
  ];
 
  if (streaming && onStreamChunk) {
    const chunks: string[] = [];
    const stream = await client.messages.stream({
      model: "claude-haiku-4-5-20251001",
      max_tokens: 256,
      system: systemConfig,
      messages: [{ role: "user", content: userInput }],
    });
 
    for await (const chunk of stream) {
      if (
        chunk.type === "content_block_delta" &&
        chunk.delta.type === "text_delta"
      ) {
        chunks.push(chunk.delta.text);
        onStreamChunk(chunk.delta.text);
      }
    }
 
    const finalMessage = await stream.finalMessage();
    return { text: chunks.join(""), usage: extractUsage(finalMessage) };
  } else {
    const response = await client.messages.create({
      model: "claude-haiku-4-5-20251001",
      max_tokens: 256,
      system: systemConfig,
      messages: [{ role: "user", content: userInput }],
    });
 
    const text =
      response.content[0].type === "text" ? response.content[0].text : "";
    return { text, usage: extractUsage(response) };
  }
}

Retry Handling

async function generateWithRetry(
  options: GenerateOptions,
  maxRetries = 3
): Promise<{ text: string; usage: UsageStats }> {
  let lastError: Error | null = null;
 
  for (let attempt = 0; attempt < maxRetries; attempt++) {
    try {
      return await generateOptimized(options);
    } catch (error) {
      lastError = error as Error;
 
      if (error instanceof Anthropic.RateLimitError) {
        const waitMs = Math.pow(2, attempt) * 1000;
        console.warn(`Rate limit hit. Retrying in ${waitMs}ms...`);
        await new Promise((resolve) => setTimeout(resolve, waitMs));
        continue;
      }
 
      if (error instanceof Anthropic.APIConnectionError) {
        await new Promise((resolve) => setTimeout(resolve, 500));
        continue;
      }
 
      throw error;
    }
  }
 
  throw lastError ?? new Error("Max retries exceeded");
}

Before and After

Absolute figures depend heavily on your usage pattern. What follows holds the token profile from the top of the article constant and computes cost from published per-MTok pricing. It isn't my invoice — it's a way to size each lever against the others.

Assumptions: 800-token system prompt, 125-token user input, 250-token output, 30,000 requests per month.

The ordering is unambiguous. Two-thirds of the total reduction came from the model swap alone. Prompt caching stacks another 23.6% on top of that — a second-stage lever, not a first one. Streaming doesn't appear in this table at all, because it moves perceived latency rather than cost.

Building this table is where I realized I had worked the problem backwards. I started with caching, the smaller lever, and I had it configured wrong on top of that.

One dated note worth planning around: Sonnet 5's introductory $2/$10 pricing ends on September 1, 2026, moving to $3/$15. If you're on Sonnet 5, the input side of this same formula multiplies by exactly 1.5. The input-price gap against Haiku 4.5 widens from 2× to 3×, which makes the "does this task actually need Sonnet?" question worth more after September than before.

Calculate the Cache Break-Even Before You Enable It

Below a 21.7% hit rate, caching costs you money

Most explanations of prompt caching say "higher hit rate, bigger savings" and stop there. The boundary that hurt me in production was the one on the other side.

The pricing works like this: writing to the cache costs 1.25× the normal input rate, and reading from it costs 0.1×. Writes are expensive; reads are how you earn it back.

For a cached block with hit rate h, the per-request coefficient is 1.25(1 - h) + 0.1h. Break-even is where that equals 1.0 — normal, uncached transmission. Solving gives h = 0.25 / 1.15, about 0.217.

Computed against the same token profile:

At a 0% hit rate, enabling caching raises the bill by 9.2%. That isn't "marginal benefit." That's a loss.

And it's a realistic scenario. The ephemeral TTL is five minutes. During any window where requests arrive less often than that, you accumulate writes and never collect reads. Split your hit rate by hour and the daytime and overnight numbers will look like they came from different applications. A single blended average hides the hours sitting below the break-even line.

Running warmup unconditionally can cost more than it saves

I ran the four-minute warmup from earlier in this article through the same math.

Firing every four minutes for thirty days is 10,800 calls per month. Each one writes 800 tokens to the cache, putting warmup alone at $11.29 per month — a fixed cost that doesn't scale down with your traffic.

Unconditional warmup only beats no caching at roughly 15,700 requests per month — about 520 a day, or one every 2.8 minutes on average. Below that, adding warmup makes your bill larger.

The sharper comparison is against a cache that's already getting natural hits. At 30,000 requests a month, unconditional warmup lands at $54.94, while no warmup with a 77.5% natural hit rate lands at $49.86. Above a 59.1% natural hit rate at that volume, warmup is dragging you backwards.

The threshold by scale:

So warmup isn't a technique that makes caching cheaper. It's a technique for filling in the quiet hours specifically. I replaced the unconditional setInterval with a check against time since the last real request.

let lastRequestAt = 0;
 
export function markRequest(): void {
  lastRequestAt = Date.now();
}
 
// Only warm the cache if no real request arrived in the last four minutes
async function warmUpIfIdle(): Promise<void> {
  const idleMs = Date.now() - lastRequestAt;
  if (idleMs < 4 * 60 * 1000) {
    return; // Natural hits are happening. Don't add a write on top.
  }
  await warmUpCache();
}
 
setInterval(warmUpIfIdle, 60 * 1000); // Check every minute, fire only when needed

What you need in order to decide is the distribution of your request intervals, not the average. Measure the percentage of requests that arrive within five minutes of the previous one. That number is approximately the ceiling on the hit rate you can reach without warmup at all.

Key Lessons

On Haiku 4.5: Front-load your critical instructions. If instruction compliance drops after switching from Sonnet, the issue is almost certainly prompt structure, not model capability.

On streaming: Don't apply it uniformly. Short outputs (<100 chars) are often better delivered all at once. Always log stop_reason — max_tokens hits in production are a warning sign that your token budget and prompt constraints are misaligned.

On prompt caching: Cache breakpoints belong on fixed content. And if your traffic is intermittent, caching can cost more than it saves — the math put the floor for unconditional warmup at roughly 15,700 requests per month. Below that, not caching is the cheaper decision.

Closing Note

Looking back, more of my time went into deciding the order of these changes than into understanding any of them individually.

Whether Haiku 4.5 is the right model isn't a question the documentation can answer. You answer it by putting outputs side by side in your own app and counting how often the result falls below what you'd ship. I deferred that check and spent the time tuning cache settings instead — the lever that turned out to be three times smaller.

The break-even was the same kind of mistake. I enabled caching on the belief that a higher hit rate simply meant more savings, and it took me a long time to notice there were hours where no hits were happening at all. Knowing the number was 21.7% would have told me immediately what to measure first.

Streaming and prompt caching are tools. Neither is the point. They exist so that someone using your app feels the answer came back quickly and that it fits what they asked.

If you're staring at an API bill that feels too high, I'd suggest this order: confirm the model matches the task by comparing outputs, then measure your request-interval distribution, and only then touch caching. That sequence would have saved me about two weeks.

The first step is small — line up ten outputs and check whether your current model is heavier than the task needs. I'm still partway through this myself; combining batch processing with the rest, once rate limits allow more headroom, is next on my list.

Related Reading

• How I Cut My Claude API Bill in Half With Prompt Caching
• Claude API Streaming & Tool Use in Production — Patterns for Parallel Calls, Error Handling, and Retry Strategies
• Four Infrastructure Levers That Cut Claude API Latency Before You Touch the Model