Make Claude Your Production Debugging Companion: A Practical Design for Log Triage, Hypothesis Generation, and Repro Scripts

Designing the log summarization prompt

The first tool is a dedicated log summarizer. The goal is not "tell me what happened in plain English" but "transform raw logs into a machine-readable intermediate representation that the next prompt can consume."

What this code does — turn raw logs into a structured timeline summary that the hypothesis prompt can ingest directly.

# log_summarizer.py
import anthropic
import json
 
client = anthropic.Anthropic()
 
LOG_SUMMARY_SYSTEM = """You are a senior SRE on first-line incident response.
Read the logs and return a structured summary as JSON.
 
Output format (strict):
{
  "incident_window": "ISO8601 range of the affected time",
  "primary_error": "one-line description of the most frequent or earliest error",
  "error_chain": [{"timestamp": "...", "service": "...", "message": "..."}, ...],
  "affected_users": "estimated impact (use 'unknown' if unclear)",
  "deduped_count": {"error_signature": count},
  "first_questions": ["up to three things a human should check next"]
}
 
Rules:
- Truncate stack traces to the top three frames.
- Replace any visible PII (emails, tokens, names) with [REDACTED].
- Use 'unknown' for fields you are not confident about. Empty strings are forbidden.
"""
 
def summarize_logs(raw_logs: str) -> dict:
    """Convert raw logs into a structured incident summary."""
    try:
        msg = client.messages.create(
            model="claude-sonnet-4-6",
            max_tokens=2000,
            system=LOG_SUMMARY_SYSTEM,
            messages=[{"role": "user", "content": raw_logs}],
        )
        text = msg.content[0].text.strip()
        # Claude sometimes wraps the JSON in code fences — strip them.
        if text.startswith("```"):
            text = text.split("```")[1]
            if text.startswith("json"):
                text = text[4:]
        return json.loads(text)
    except json.JSONDecodeError as e:
        return {"_raw": msg.content[0].text, "_parse_error": str(e)}
    except anthropic.APIError as e:
        return {"_api_error": str(e)}
 
# Expected output
# {
#   "incident_window": "2026-04-29T02:14:00Z/2026-04-29T02:18:00Z",
#   "primary_error": "Stripe webhook signature verification failed",
#   "error_chain": [...],
#   "affected_users": "estimated 12 paid users",
#   "deduped_count": {"InvalidSignature": 47, "TimeoutError": 3},
#   "first_questions": [
#     "Did STRIPE_WEBHOOK_SECRET change in the last deploy?",
#     "Is the request timestamp header drifting from server time?",
#     "Could the Cloudflare layer be mutating the request body?"
#   ]
# }

Why JSON instead of prose? Because the next prompt needs to consume this output mechanically. If you accept free-form text, the format drifts every run and your automation breaks.

The first_questions array is capped at three on purpose. A human under stress can only juggle a handful of branches at once; ten suggestions would just get ignored from the fourth onward. Truncating stack traces to three frames is a similar guardrail — long stacks crowd out the rest of the context window. If the bottom of the stack actually matters, the verification phase pulls it in on demand.

Hypothesis-driven prompting

Once you have a clean summary, the next move is to generate hypotheses with a verification plan attached. A bare list of "things that might be wrong" is not enough; you want each hypothesis paired with a concrete way to confirm it.

What this code does — convert a structured summary into a prioritized list of hypotheses, each with an associated verification action.

# hypothesis_generator.py
import anthropic
import json
 
client = anthropic.Anthropic()
 
HYPOTHESIS_SYSTEM = """You are a senior engineer who runs first-line incident response.
Given a structured incident summary, produce hypotheses for the root cause.
 
Output format (JSON):
{
  "hypotheses": [
    {
      "id": "h1",
      "summary": "one-line hypothesis",
      "likelihood": "high|medium|low",
      "evidence_in_logs": ["which parts of the summary support this"],
      "verification": {
        "type": "command|query|manual",
        "action": "the exact command or check to run",
        "expected_if_true": "what you would see if this hypothesis is correct"
      },
      "rollback_safe": true
    }
  ],
  "recommended_first": "h1",
  "do_not_jump_to": "any taboo actions in this incident — DB writes, blind rollbacks, etc."
}
 
Rules:
- At most four hypotheses.
- Each hypothesis must be independently verifiable.
- Verification commands must be read-only by default.
- Set rollback_safe to false if the fix involves DB migrations or data deletion.
"""
 
def generate_hypotheses(summary: dict, app_context: str) -> dict:
    user_msg = f"""# App context
{app_context}
 
# Incident summary
{json.dumps(summary, ensure_ascii=False, indent=2)}
 
Return hypotheses for first-line response in the JSON format above.
"""
    msg = client.messages.create(
        model="claude-sonnet-4-6",
        max_tokens=2500,
        system=HYPOTHESIS_SYSTEM,
        messages=[{"role": "user", "content": user_msg}],
    )
    text = msg.content[0].text.strip()
    if text.startswith("```"):
        text = text.split("```")[1]
        if text.startswith("json"):
            text = text[4:]
    return json.loads(text)

The hidden virtue of this prompt is the verification.type enum. If you let Claude write free-form verification steps, it will cheerfully suggest things like "just edit the production database to fix this row." Splitting verification into command | query | manual makes the danger level legible at a glance, and lets you wire automation safely (commands are auto-runnable, manual checks need a human).

The do_not_jump_to field exists for a specific reason: I once almost rolled back a deploy that contained a database migration, which would have left orphaned rows. Claude saved me by flagging "this commit includes a migration; rolling back will leave inconsistent state." It is worth the few extra tokens.

Generating minimal repro scripts

Once one hypothesis looks promising, you want to confirm it without touching production. Claude is excellent at generating minimal repro scripts when prompted carefully.

What this code does — take a single hypothesis as input and produce a self-contained, locally runnable script along with expected output.

# repro_generator.py
import anthropic
 
client = anthropic.Anthropic()
 
REPRO_SYSTEM = """You are an expert at writing minimal reproductions of production bugs.
Given a hypothesis and the relevant tech stack, produce a script someone can run locally.
 
Output format:
1. ## Prerequisites
   environment variables and dependencies needed
2. ## Repro script
   ```python (or shell) code fence with the actual script
3. ## Expected output
   what the run should print if the hypothesis is correct, and what to expect otherwise
4. ## Safety
   why this script will not affect production, or warnings if it might
 
Rules:
- Never write to the production database.
- If the script calls external APIs, keep the call count to one to three.
- If a mock server is needed, include the steps to start it.
"""
 
def generate_repro(hypothesis: dict, tech_stack: str) -> str:
    user_msg = f"""# Tech stack
{tech_stack}
 
# Hypothesis to test
{hypothesis}
 
Write a minimal local reproduction.
"""
    msg = client.messages.create(
        model="claude-sonnet-4-6",
        max_tokens=3000,
        system=REPRO_SYSTEM,
        messages=[{"role": "user", "content": user_msg}],
    )
    return msg.content[0].text

For my own apps, I save these generated scripts into scripts/repro/ and check them into the repo. They double as regression test starting points and as institutional memory — when a similar incident happens months later, I can search by symptom and find the script that proved or refuted a similar hypothesis.

To raise the quality further, pin the tech stack into a Claude Project knowledge base so that you do not have to paste it on every invocation. This also makes the repro scripts feel idiomatic to your codebase rather than generic Python.

Sanitizing logs before they leave your infrastructure

Up to this point I have been waving "send logs to Claude" around casually, but production logs contain emails, tokens, and other PII you really should not be exporting to a third party. Treat sanitization as non-negotiable — both for legal reasons and because once a token leaks into someone else's training corpus, you cannot pull it back.

What this code does — a small, extensible sanitization pipeline that runs before any log is sent to the API.

# log_sanitizer.py
import re
from typing import Pattern
 
SENSITIVE_PATTERNS: list[tuple[Pattern[str], str]] = [
    # email addresses
    (re.compile(r"[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}"), "[EMAIL]"),
    # JWT (three base64url segments)
    (re.compile(r"eyJ[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+"), "[JWT]"),
    # Stripe keys
    (re.compile(r"sk_(live|test)_[A-Za-z0-9]{16,}"), "[STRIPE_KEY]"),
    # GitHub PATs
    (re.compile(r"ghp_[A-Za-z0-9]{20,}"), "[GH_PAT]"),
    # Anthropic API keys
    (re.compile(r"sk-ant-[A-Za-z0-9_-]{20,}"), "[ANTHROPIC_KEY]"),
    # 13–19 digit number sequences that look like card numbers
    (re.compile(r"\b(?:\d[ -]*?){13,19}\b"), "[CC_NUMBER]"),
    # IPv4
    (re.compile(r"\b(?:\d{1,3}\.){3}\d{1,3}\b"), "[IP]"),
]
 
def sanitize_log(log: str, max_lines: int = 300) -> str:
    """Apply masking patterns and clip the log to a manageable size."""
    redacted = log
    for pattern, replacement in SENSITIVE_PATTERNS:
        redacted = pattern.sub(replacement, redacted)
 
    lines = redacted.splitlines()
    if len(lines) > max_lines:
        head = lines[: max_lines // 2]
        tail = lines[-max_lines // 2 :]
        skipped = len(lines) - len(head) - len(tail)
        return "\n".join(head + [f"... ({skipped} lines skipped) ..."] + tail)
    return redacted
 
if __name__ == "__main__":
    sample = """
[2026-04-29T02:14:00Z] ERROR user=alice@example.com token=eyJhbGciOiJIUzI1NiJ9.payload.sig
[2026-04-29T02:14:01Z] Stripe key sk_live_AbCdEf1234567890abcdef leaked!
""".strip()
    print(sanitize_log(sample))
    # Expected output:
    # [2026-04-29T02:14:00Z] ERROR user=[EMAIL] token=[JWT]
    # [2026-04-29T02:14:01Z] Stripe key [STRIPE_KEY] leaked!

The sanitization pass should run twice in your pipeline: once when logs are written (so they are clean at rest), and again right before sending them to Claude (defense in depth). In practice the second pass catches things like Stripe webhook bodies, where a token can sneak in through fields you did not anticipate.

The 300-line cap is empirical. Beyond that, summarization quality drops because Claude starts skipping the tail. If a longer window is genuinely needed, run the summarizer first and feed the compacted output forward — that scales much better than throwing everything at one prompt.

Traps I keep seeing (and how to avoid them)

These are the patterns I have stepped in personally and now actively avoid.

1. Do not paste "all the logs."

I once dumped 100,000 lines of production logs and asked Claude to figure it out. The bill was painful and the answer was vague. Always run the summarization step first; raw logs are a supporting reference, not the main input.

2. Do not trust hypotheses without verification.

Treat each hypothesis like the first guess of a junior engineer who just walked into the war room. The reasoning is usually plausible, but the model has no idea about your code's quirks. Always run verification.action before acting on a hypothesis.

3. Do not let the agent commit fixes during the first thirty minutes.

Letting Claude Code commit code on your behalf is great in normal development. During an active incident, it creates meta-incidents — "wait, did the bot just push something while I was investigating?" Keep diagnosis and remediation separate, and gate any agent-driven commit behind explicit human approval.

4. Do not let it casually recommend rollbacks.

The do_not_jump_to field exists because Claude, given "errors started after the latest deploy," loves to suggest rollbacks. But rollbacks across migrations are dangerous. Asking the prompt to evaluate rollback_safe explicitly cuts the impulse before it reaches a tired human.

5. Do not blur service boundaries.

When you concatenate logs from multiple services, mark the seams clearly:

=== Service: stripe-webhook-worker ===
[2026-04-29T02:14:00Z] ...
 
=== Service: nextjs-app ===
[2026-04-29T02:14:01Z] ...

Without these dividers, Claude has to guess which line belongs to which service and tends to invent causality across them. Explicit boundaries are the cheapest accuracy upgrade you will ever apply.

Wiring it into a real SaaS

To close the loop, here is roughly how I run this in a small SaaS I operate solo. Sentry and Cloudflare Logpush funnel notable errors into a Cloudflare Queue. A Worker pulls messages in batches, runs the pipeline, and posts the result to a dedicated Slack channel.

// queue_handler.ts
export default {
  async queue(batch: MessageBatch<ErrorEvent>, env: Env) {
    const grouped = groupByService(batch.messages);
 
    for (const [service, events] of grouped) {
      const rawLogs = events.map(e => e.body).join("\n");
      const sanitized = sanitizeLog(rawLogs); // TS port of log_sanitizer
      const summary = await summarizeLogs(env.ANTHROPIC_API_KEY, sanitized);
 
      // Hypothesis generation only for high-severity batches (cost control).
      const isHigh = events.some(e => e.body.includes("CRITICAL"));
      const hypotheses = isHigh
        ? await generateHypotheses(env.ANTHROPIC_API_KEY, summary, APP_CTX)
        : null;
 
      await postToSlack(env.SLACK_WEBHOOK, formatReport(service, summary, hypotheses));
    }
  },
};

Two implementation notes worth taking. First, prompt caching is essential here. The system prompts above are long, and caching them shaves thousands of tokens off every call. Without it, the bill grows quickly with every additional service routed through the queue.

Second, the isHigh gate exists because running hypothesis generation on every error is rarely worth it. Most batches just want a Slack-friendly summary; only a small fraction deserves the full hypothesis-and-verification treatment. Letting humans pull the trigger for deeper analysis keeps the system economically sane.

A real incident: the Stripe webhook timeout

Let me ground this with a story. A few months ago my SaaS started rejecting webhook deliveries from Stripe at about 4 a.m. Sunday. The pager fired, and Sentry showed roughly fifty InvalidSignature errors clustered in a four-minute window, with no other obvious activity around them.

Without the pipeline I would have spent twenty minutes squinting at logs trying to remember the exact verification flow. Instead, I dropped the raw log batch into the sanitizer, ran the summarizer, and got back a JSON object whose first_questions were:

• "Did STRIPE_WEBHOOK_SECRET change in the last deploy?"
• "Is the request timestamp drifting?"
• "Is something between Stripe and the worker rewriting the body?"

Those three questions were exactly the right ones. The hypothesis generator then ranked "Cloudflare-side body transformation" as medium, because the summary mentioned a recent change to a custom worker route. I ran the verification command (a curl reproducing a known-good payload through the same route), got an unexpected JSON re-serialization, and shipped a minimal fix in about twelve minutes from pager to PR.

The point is not that Claude solved the bug — I solved it. But Claude compressed the staring phase from "twenty minutes of bleary scanning" to "ninety seconds of reading three crisp questions." That compression is the entire value proposition.

Choosing the right model and being honest about cost

It is tempting to default to the most powerful model for everything. In practice I run a tiered setup, because cost adds up quickly when a queue is firing.

• Summarization — claude-sonnet-4-6 is the sweet spot. Haiku-class models occasionally drop fields from the JSON schema, and Opus is overkill for what is essentially structured extraction. Pair Sonnet with prompt caching and you will pay a small fraction of what an unbatched flow costs.
• Hypothesis generation — same model. The reasoning required is comparable to a thoughtful junior engineer, not a research-grade analyst.
• Repro generation — Sonnet again. If you find Claude struggling with the most exotic parts of your stack, I would experiment with bigger context first (paste the relevant module verbatim) before reaching for a heavier model.
• Confirmation and write-ups — sometimes I escalate to Opus when I want a richer postmortem narrative for the team channel. That happens after the incident is contained, not during.

A hard lesson I had to learn was to instrument the cost. Without an explicit dashboard, the API spend can balloon silently. I now log per-incident token usage to a tiny KV counter, with a daily cap that pages me if it is exceeded. The cap saved me twice when a misconfigured queue retried a message in a tight loop and would have consumed thousands of dollars overnight if left unchecked.

Boundary lines: what the agent should never own

Even after using this setup for many months, I am fairly conservative about what I let Claude do unsupervised. Here is the boundary I draw, written down so I do not erode it under pressure.

• Reading and summarizing logs: yes, fully delegated.
• Proposing hypotheses with verification: yes, fully delegated.
• Running read-only verification commands automatically: only behind a command allowlist, and only inside an isolated sandbox.
• Touching production state: never. Database writes, feature flag flips, and traffic shifts go through me, even at 3 a.m.
• Posting to public status pages: never. Customer-facing communication is one of the few places I personally review every word, because tone matters and Claude's defaults can read as either too breezy or too alarming.
• Closing or merging PRs: only after I have read the diff. I will let Claude open a PR with proposed changes, but the merge button stays human-controlled until further notice.

The drift to watch for is "feature creep at 2 a.m." After a successful triage, it is tempting to think, "great, next time I will let it just go ahead and apply the fix." Resist that urge until you have seen the system handle at least a dozen incidents reliably. The cost of one wrong autonomous action — a bad migration rollback, a flag flipped on a customer's behalf — is much higher than the value of saving a few minutes per incident.

Iterating on the prompts: how I keep them sharp

The prompts above are a snapshot of where I am right now. They were not perfect on day one. I keep them in a small repository with a changelog so that I can see the trajectory and roll back when an "improvement" turns out to be a regression.

A useful habit is the post-incident prompt review. After each non-trivial incident, I open the prompt files and ask: did the summarizer surface the right first_questions? Did the hypothesis generator over-rank a low-likelihood guess? Did the repro script touch anything it should not have? If the answer to any of these is "no," I write a tiny test case using the actual sanitized log from that incident and re-run the prompt against the candidate change. The test case lives in the repo as a regression fixture.

Another habit worth mentioning is templating the change history into the system prompts themselves. When the summarizer prompt rejects an output format that worked last week, I want to know why. Adding a one-line "version" field at the bottom of the system prompt — even just # v3, last edited 2026-04-20 — has saved me from chasing phantom regressions multiple times.

Finally, do not be afraid to write site-specific instructions. Generic best practices only go so far. The summarizer in my Stripe-heavy app explicitly says "treat 'InvalidSignature' as a top-priority error category, even if rare in volume," because that pattern matters more for revenue than the same number of generic 500s. Tailoring the prompt to your business will do more for accuracy than any model upgrade.

Sharing the load with future-you

One last reflection. The deepest reason I built this pipeline is not "Claude is fast" or even "tokens are cheap." It is that incident response, done alone, is a particular kind of lonely. You are tired, the customer is upset, and the only feedback loop you have is your own internal monologue.

A well-designed companion does not just save time — it externalizes the loop. When the summary comes back as a tidy JSON object, it acts as proof that someone (or something) else has at least looked at the problem with you. The hypothesis prompt is, in effect, a small ritual of "let us list out what could be wrong before we panic." The repro script is "let us reach for the keyboard with a plan, not a hunch."

If you carry the pager alone, this kind of structure is more important than any individual prompt. It is the scaffolding that lets you behave like a professional even when your sleep-deprived brain wants to thrash. Build it once, refine it over time, and it will outlast any specific outage. That, more than the raw productivity gains, is what I would want a fellow solo developer to take from this article.

What to do tomorrow morning

You do not need to build the whole pipeline at once. The single most useful step is to copy the sanitizer, run your usual logs through it, and look at how much PII falls out. That alone changes how you think about pasting production data into chats.

After that, wait until your next real incident and try the summarizer prompt instead of free-form chat. If you find yourself wishing for an immediate first cut of hypotheses, add the second prompt. Layer the rest as the pain demands. That order — pain-driven adoption — is how this stack stayed maintainable for me.

I still dislike pager nights. But having Claude in front of the dashboard with me makes the loneliness more manageable. If even one corner of this article saves you twenty minutes the next time your phone buzzes at 2 a.m., it has earned its keep.

If you want to deepen the underlying mental model, I recommend Practical Monitoring by Mike Julian (O'Reilly) (the Japanese edition is very good). Its layered framing of "symptom / cause / business impact" is exactly the lens I would use to refine the summarization prompt above.