Production-Grade Hallucination Defense for Claude API: A Multi-Layer Architecture

Layer 2: Tool Use as an escape hatch

Grounding alone does not stop reasoning errors — Claude can still misread the documents. The fix is to give Claude a structured way to declare "I cannot answer," using Tool Use.

const escapeHatchTools: Anthropic.Tool[] = [
  {
    name: "answer_with_citation",
    description:
      "Answer the user's question using the supplied documents as the only source. " +
      "You must include the IDs of the documents you cited.",
    input_schema: {
      type: "object" as const,
      properties: {
        answer: { type: "string", description: "Answer text shown to the user" },
        citations: {
          type: "array",
          items: { type: "string" },
          description: "Document IDs used as evidence (at least one required)",
        },
        confidence: {
          type: "string",
          enum: ["high", "medium", "low"],
          description: "Your confidence in this answer",
        },
      },
      required: ["answer", "citations", "confidence"],
    },
  },
  {
    name: "decline_to_answer",
    description:
      "Use this when the documents do not contain enough information, when the question is " +
      "out of scope, or when it is ambiguous. Do not guess instead of calling this tool.",
    input_schema: {
      type: "object" as const,
      properties: {
        reason: {
          type: "string",
          enum: ["insufficient_information", "out_of_scope", "ambiguous_question"],
        },
        suggested_action: {
          type: "string",
          description:
            "An alternative to suggest to the user (e.g., contact a human, provide more detail)",
        },
      },
      required: ["reason", "suggested_action"],
    },
  },
];
 
async function answerWithEscapeHatch(question: string, docs: RetrievedDoc[]) {
  const docsBlock = docs
    .map((d) => `<document id="${d.id}">${d.content}</document>`)
    .join("\n");
 
  const res = await client.messages.create({
    model: "claude-sonnet-4-5",
    max_tokens: 1024,
    tools: escapeHatchTools,
    tool_choice: { type: "any" }, // Force tool use
    system:
      "If you can answer, call answer_with_citation. If you cannot, call decline_to_answer. " +
      "Do not return plain text under any circumstances.",
    messages: [
      { role: "user", content: `<documents>\n${docsBlock}\n</documents>\n\n${question}` },
    ],
  });
 
  const toolUse = res.content.find(
    (b): b is Anthropic.ToolUseBlock => b.type === "tool_use"
  );
  if (!toolUse) {
    throw new Error("Claude returned no tool call - check tool_choice config");
  }
  return { tool: toolUse.name, input: toolUse.input };
}

Why does this help so much more than the same instruction in plain prose? When you ask Claude to "say I don't know," compliance is a soft preference. When you make it call a decline_to_answer tool with a typed schema, refusal becomes an action with structured arguments. Claude resists the pull toward giving a vague answer because the alternative path is a concrete first-class action it has been told to use.

The implementation detail that matters most is tool_choice: { type: "any" }. Without it, Claude decides on its own whether to use a tool or return plain text, and you will see plain-text leakage. In my production traffic this single line moved violation rates by an order of magnitude.

Making the confidence field required is also deliberate. Once Claude has to declare its own confidence, you can branch on low to add a "please confirm with us if this is critical" footer, or to escalate to a human reviewer.

Layer 3: Citations API to mechanically force evidence

Anthropic's Citations feature makes Claude attach the exact source span for each sentence it generates. Because Claude must point at real text in the input, fabrication of strings that are not in the source becomes structurally harder.

async function answerWithCitations(question: string, docs: RetrievedDoc[]) {
  const documentBlocks = docs.map((d) => ({
    type: "document" as const,
    source: {
      type: "text" as const,
      media_type: "text/plain" as const,
      data: d.content,
    },
    title: d.title,
    citations: { enabled: true },
  }));
 
  const res = await client.messages.create({
    model: "claude-sonnet-4-5",
    max_tokens: 1024,
    messages: [
      {
        role: "user",
        content: [...documentBlocks, { type: "text", text: question }],
      },
    ],
  });
 
  // Each text block has a citations array attached
  const annotated = res.content.flatMap((block) => {
    if (block.type !== "text") return [];
    return [
      {
        text: block.text,
        citations: block.citations ?? [],
      },
    ];
  });
 
  // Segments with no citation are high hallucination risk — flag them
  for (const seg of annotated) {
    if (seg.text.trim().length > 30 && seg.citations.length === 0) {
      console.warn(`Uncited segment: "${seg.text.slice(0, 50)}..."`);
    }
  }
 
  return annotated;
}

When citations are enabled, Claude experiences a cognitive constraint: every sentence needs a pointer to source text. After generation, cited_text (the exact original string) is attached to each segment, which means you can apply mechanical post-filters such as "drop any segment whose citations array is empty" or "reject any number that doesn't appear in the cited text."

Layer 4: Self-consistency to weed out unstable answers

This is where the production-grade work begins. Send the same question 3–5 times and check whether the answer is stable. Hallucinations are stochastic, so they tend to vary between samples; if Claude returns the same fact on every run, your trust in it is much higher.

import crypto from "node:crypto";
 
interface SelfConsistencyResult {
  consensus_answer: string | null;
  agreement_rate: number;
  variations: string[];
}
 
async function selfConsistencyAnswer(
  question: string,
  docs: RetrievedDoc[],
  samples = 5
): Promise<SelfConsistencyResult> {
  const docsBlock = docs.map((d) => `<doc id="${d.id}">${d.content}</doc>`).join("\n");
 
  // Raise temperature to inject diversity, then check whether answers still converge
  const responses = await Promise.all(
    Array.from({ length: samples }, () =>
      client.messages.create({
        model: "claude-sonnet-4-5",
        max_tokens: 256,
        temperature: 0.7,
        system:
          "Extract one factual sentence from the supplied documents. " +
          "No preamble, no explanation, just the fact.",
        messages: [
          { role: "user", content: `<docs>${docsBlock}</docs>\n\nQuestion: ${question}` },
        ],
      })
    )
  );
 
  const variations = responses.map((r) =>
    r.content
      .filter((b): b is Anthropic.TextBlock => b.type === "text")
      .map((b) => b.text.trim())
      .join("")
  );
 
  // Group by exact normalized match (use embedding clustering for production)
  const counts = new Map<string, number>();
  for (const v of variations) {
    const key = normalize(v);
    counts.set(key, (counts.get(key) ?? 0) + 1);
  }
 
  const sorted = Array.from(counts.entries()).sort((a, b) => b[1] - a[1]);
  const top = sorted[0];
 
  return {
    consensus_answer: top && top[1] >= Math.ceil(samples / 2) ? top[0] : null,
    agreement_rate: top ? top[1] / samples : 0,
    variations,
  };
}
 
function normalize(s: string): string {
  return s
    .replace(/\s+/g, " ")
    .replace(/[.,;:!?'"()\[\]]/g, "")
    .toLowerCase();
}

Exact match will not line up — what running it showed

The normalize function above does not absorb paraphrases. I ran five sentences stating the same fact through it in Node.js (hand-written paraphrases as mock input, not real Claude calls).

const v = [
  "Cancellation is fully refundable within 30 days of purchase.",
  "Cancel within 30 days of purchase for a full refund.",
  "You can cancel within 30 days and get a full refund.",
  "Cancellation is fully refundable within 30 days of purchase",
  "Cancellation is fully refundable within 14 days of purchase.", // the one that differs
];
// exact match after normalize:  { rate: 0.4, groups: 4 }
// key built from numbers/units: { rate: 0.8, groups: 2 }

Four of the five agree on the fact, yet the exact-match agreement_rate is 0.4 — below the 0.6 threshold. A small change in wording is enough to make a stable answer look unstable. Self-consistency should compare facts, not surface text.

Switching the comparison unit to the numbers, dates and proper nouns in the answer gives 0.8 on the same input, and the single disagreeing answer (14 days) stays in the minority.

This has a limit too. "Fully refundable within 30 days" and "Not refundable even within 30 days" contain the same number, so they land in the same group (the check returned rate 1). Reversals of polarity cannot be caught by a number key. Leave those to the Layer 5 judge, or to a design that narrows the answer to a yes/no choice.

// Minimal version: extract only numbers, dates and amounts as the fact key
function factKey(s: string): string {
  const nums = s.match(/\d+(?:[.,]\d+)?\s*(?:days?|%|USD|JPY|hours?|items?)?/g) ?? [];
  return nums.map((x) => x.replace(/\s/g, "")).sort().join("|");
}

Covering proper nouns needs tokenization or embedding clustering. To start, narrow the question to a number, a yes/no, or a fixed set of options before running self-consistency; the threshold becomes much easier to read.

In practice I treat agreement_rate >= 0.6 as the trust threshold. Below that, the system either replies "this answer was unstable, please confirm with a human" or escalates to Opus.

The trade-off is obvious: 5x the API calls and significantly higher latency. In production I only apply self-consistency to the small slice of traffic where errors are expensive — high-stakes pricing displays, medical or legal information, and similar. That slice is around 5% of total requests in my system.

Layer 5: LLM-as-judge for independent verification

This is the strongest single move in the playbook. After generating an answer, pass it to a separate Claude session — a fresh system prompt, a clean conversation — and ask whether the answer is supported by the source documents.

interface JudgeVerdict {
  verdict: "supported" | "unsupported" | "partially_supported";
  reasoning: string;
  problematic_claims: string[];
}
 
async function judgeAnswer(
  question: string,
  answer: string,
  source_docs: RetrievedDoc[]
): Promise<JudgeVerdict> {
  const docsBlock = source_docs.map((d) => `[${d.id}] ${d.content}`).join("\n\n");
 
  // Make the judge adversarial. A friendly judge defeats the entire layer.
  const judgeSystem = `You are a strict factual auditor. You evaluate whether an AI's answer is genuinely supported by the supplied source documents.
 
Criteria:
- Proper nouns, numbers, dates, and URLs in the answer must appear verbatim in the sources to count as supported
- "Common-sense correct" or "reasonable inference" never counts as supported
- If part of the answer is correct but other parts include unsupported information, mark partially_supported
- Even if the answer looks fine, list any subtle inaccuracies you can find
 
For each problem, add an entry to problematic_claims as "Claim: Issue".`;
 
  const res = await client.messages.create({
    model: "claude-opus-4-6", // Use a strong model for the judge
    max_tokens: 1024,
    system: judgeSystem,
    tools: [
      {
        name: "submit_verdict",
        description: "Submit the factual verification verdict",
        input_schema: {
          type: "object" as const,
          properties: {
            verdict: {
              type: "string",
              enum: ["supported", "unsupported", "partially_supported"],
            },
            reasoning: { type: "string" },
            problematic_claims: { type: "array", items: { type: "string" } },
          },
          required: ["verdict", "reasoning", "problematic_claims"],
        },
      },
    ],
    tool_choice: { type: "tool", name: "submit_verdict" },
    messages: [
      {
        role: "user",
        content:
          `Sources:\n${docsBlock}\n\n` +
          `User question: ${question}\n\n` +
          `AI answer: ${answer}\n\n` +
          `Is this answer supported by the sources? Apply the criteria strictly.`,
      },
    ],
  });
 
  const tool = res.content.find(
    (b): b is Anthropic.ToolUseBlock => b.type === "tool_use"
  );
  if (!tool) throw new Error("Judge did not return a verdict");
  return tool.input as unknown as JudgeVerdict;
}
 
// Pipeline composition
async function safeAnswer(question: string, docs: RetrievedDoc[]) {
  const draft = await answerWithGrounding(question, docs);
  const verdict = await judgeAnswer(question, draft, docs);
 
  if (verdict.verdict === "unsupported") {
    return {
      shown_to_user:
        "I am sorry — I do not have enough verified information to answer that confidently.",
      internal_log: { draft, verdict },
    };
  }
  if (verdict.verdict === "partially_supported") {
    const refined = await refineWithFeedback(draft, verdict.problematic_claims, docs);
    return { shown_to_user: refined, internal_log: { draft, verdict, refined } };
  }
  return { shown_to_user: draft, internal_log: { draft, verdict } };
}
 
async function refineWithFeedback(
  draft: string,
  issues: string[],
  docs: RetrievedDoc[]
): Promise<string> {
  const docsBlock = docs.map((d) => `[${d.id}] ${d.content}`).join("\n");
  const res = await client.messages.create({
    model: "claude-opus-4-6",
    max_tokens: 1024,
    system:
      "Rewrite the draft so that every claim is supported by the supplied sources. " +
      "Return only the rewritten body.",
    messages: [
      {
        role: "user",
        content: `Sources: ${docsBlock}\n\nDraft: ${draft}\n\nIssues:\n${issues.join("\n")}`,
      },
    ],
  });
  return res.content
    .filter((b): b is Anthropic.TextBlock => b.type === "text")
    .map((b) => b.text)
    .join("");
}

The reason this works is mostly social: when the same model both generates and evaluates, a confirmation bias creeps in and verdicts soften. Use a separate session with a separate system prompt, ideally a stronger model (Sonnet for generation, Opus for the judge). After I added this loop, hallucination-driven complaints in my system dropped noticeably (I did not keep a strict count). Costs went up by 2–3x, but compared to the cost of incident response and customer apology emails, it is dramatically cheaper.

Layer 6: Schemas and types to enforce shape

For instruction-skipping hallucinations — "I asked for JSON, I got prose" — combine Tool Use schemas with a runtime validator like Zod.

import { z } from "zod";
 
const InvoiceExtractionSchema = z.object({
  invoice_number: z.string().regex(/^[A-Z0-9-]+$/),
  total_amount: z.number().positive(),
  currency: z.enum(["JPY", "USD", "EUR"]),
  issued_date: z.string().regex(/^\d{4}-\d{2}-\d{2}$/),
  line_items: z
    .array(
      z.object({
        description: z.string().min(1),
        quantity: z.number().int().positive(),
        unit_price: z.number().nonnegative(),
      })
    )
    .min(1),
});
 
type Invoice = z.infer<typeof InvoiceExtractionSchema>;
 
async function extractInvoice(rawText: string, maxRetries = 2): Promise<Invoice> {
  let attempts = 0;
  let lastError: string | undefined;
 
  while (attempts <= maxRetries) {
    const res = await client.messages.create({
      model: "claude-sonnet-4-5",
      max_tokens: 2048,
      tools: [
        {
          name: "submit_invoice",
          description: "Submit structured invoice data extracted from the text",
          input_schema: {
            type: "object" as const,
            properties: {
              invoice_number: { type: "string" },
              total_amount: { type: "number" },
              currency: { type: "string", enum: ["JPY", "USD", "EUR"] },
              issued_date: { type: "string" },
              line_items: {
                type: "array",
                items: {
                  type: "object",
                  properties: {
                    description: { type: "string" },
                    quantity: { type: "number" },
                    unit_price: { type: "number" },
                  },
                  required: ["description", "quantity", "unit_price"],
                },
              },
            },
            required: [
              "invoice_number",
              "total_amount",
              "currency",
              "issued_date",
              "line_items",
            ],
          },
        },
      ],
      tool_choice: { type: "tool", name: "submit_invoice" },
      messages: [
        {
          role: "user",
          content:
            (lastError
              ? `Previous error: ${lastError}\nFix it and try again.\n\n`
              : "") + `Invoice text:\n${rawText}`,
        },
      ],
    });
 
    const tool = res.content.find(
      (b): b is Anthropic.ToolUseBlock => b.type === "tool_use"
    );
    if (!tool) {
      lastError = "no tool call";
      attempts++;
      continue;
    }
 
    const parsed = InvoiceExtractionSchema.safeParse(tool.input);
    if (parsed.success) return parsed.data;
 
    lastError = parsed.error.issues
      .map((i) => `${i.path.join(".")}: ${i.message}`)
      .join("; ");
    attempts++;
  }
 
  throw new Error(
    `Schema validation failed after ${maxRetries + 1} attempts: ${lastError}`
  );
}

The non-obvious detail is feeding the previous error back into the next attempt. Once Claude knows what failed, it usually corrects on the next try. In my extraction task, allowing up to three retries left very few cases that still failed validation. The rate depends on the material and the schema, so measure it on your own data. JSON Schema's enum, required, and regex are good but not enough — Zod's runtime checks catch cases like 2026-13-45 where the format matches but the value is impossible.

Production pitfalls I have actually hit

Once you have the theory in place, you will encounter a specific set of issues during operation. Knowing them ahead of time saves days of debugging.

Judges go soft over time. Telling the judge "be strict" rarely lasts. Add specific phrases like "find faults from an adversarial perspective" and "use partially_supported aggressively." I sample 10 judge verdicts per week against human review, and rewrite the judge prompt the moment the gap widens.

Citations API can return empty cited_text. This happens with very long documents or when the cited fragment falls inside the middle of a sentence. Always log uncited segments and route them to human review.

All five self-consistency samples disagree. This is usually not a hallucination signal — it is a question that is too ambiguous to answer. In that case, ask the user a clarifying question before generating a final answer.

Complex Tool Use schemas invite fabrication. If your schema has more than three nested levels or more than ten required fields, Claude fabricates values to satisfy the structure. Keep schemas simple enough to scan in 30 seconds, and break complex extraction into two tool calls.

Judge cost balloons. Using Opus for every judge call is expensive. I ended up with a tiered filter: Sonnet generates, Sonnet judges, and Opus only re-judges the ones marked unsupported. Keeping Opus to under 10% of traffic kept quality high while staying inside the cost budget.

Prompt caching does not play nicely with the judge. Judge inputs change every request because they include the generated answer. You can still cache the source documents portion. See Cutting your monthly Claude API bill in half with prompt caching for the implementation details.

Thresholds are domain-specific. The right agreement_rate and the right tolerance for partially_supported vary by use case. Medical advice still struggles at 0.9; a casual chatbot is fine at 0.5. Run a 100-sample manual evaluation in your domain before going live and pick thresholds from data, not hope.

Observability without which you are flying blind

A defensive architecture is only as good as the signals you collect about it in production. Once the layers are wired up, the next step is to make their behavior visible. I instrument three categories of signals on every request, and I would not run any of these defenses without them.

The first category is per-layer rejection rates. For each layer, I log how often it triggered: how often grounding refused to answer, how often the escape hatch fired, what fraction of segments came back uncited, and how often the judge marked an answer unsupported. These numbers should not drift suddenly. A jump in escape-hatch usage often means an upstream retrieval pipeline degraded; a jump in judge rejections often means the model was updated and your prompts need to follow.

The second category is end-to-end latency by layer. Self-consistency adds 3–5x latency, and the judge typically adds another 1.5x because of the larger model. If you do not break this down per layer, you will spend hours debugging a slow request without realizing the judge is the bottleneck. I emit a span per layer using OpenTelemetry, then keep p50, p95, and p99 dashboards per route.

The third category is human-review sampling. Even with all the automated layers, I sample roughly 1 in 200 production answers for human evaluation. The cost is small, and the value is enormous: this is the only way to detect "the judge has gone soft" or "the grounded answer is technically correct but unhelpful in tone." The sampled human verdicts also feed directly into a regression set, which I rerun against new model versions and prompt edits before promoting them.

If you only adopt one observability practice from these three, choose human-review sampling. Automated metrics tell you that something changed; humans tell you what to do about it.

Cost realism: what these layers actually buy you

A common reaction to this architecture is "this looks expensive." It is. Across all six layers, my high-stakes traffic costs roughly 4–6x what the same request would cost as a single Claude call. So the question is not "can I afford the layers?" but "what does each layer buy me, and is the price right?"

Layer 1 (grounding) is essentially free. It is just prompt design, and it pays for itself within a few requests by reducing fabrication.

Layer 2 (escape hatch) adds zero API cost and one tool-use turn of latency. The improvement in user trust — because Claude now declines instead of inventing — is the highest-ROI move on the entire list.

Layer 3 (Citations API) adds a small token overhead and is essentially free at scale. Use it on any request where the source documents are non-trivial.

Layer 6 (schemas) costs almost nothing in the steady state. The retry path doubles cost only when validation fails, which should be rare if your schema is well-designed.

Layers 4 (self-consistency) and 5 (judge) are where real money goes. Self-consistency multiplies generation cost by the number of samples, and the judge adds its own full inference. This is exactly why you must apply them only to the slice of traffic where the cost of a wrong answer exceeds the cost of the verification calls.

In practice I price requests against an "incident-equivalent" — the average cost of a customer apology, refund, or human-handled escalation. If a route's hallucination cost would be $50 per incident at a 1% error rate, $0.40 of extra inference per request is a bargain.

How to choose the combination

Applying all six layers to every request explodes cost. In my production system I tier them by request criticality.

For low-stakes traffic (FAQs, casual chat), Layers 1 (grounding) and 2 (escape hatch) are enough. For mid-stakes (product specs, procedural guidance), I add Layer 3 (Citations) and Layer 6 (schemas). For high-stakes domains (medical, legal, financial), I stack Layer 4 (self-consistency) and Layer 5 (judge) on top of everything else.

For broader patterns on production design and the monitoring dashboards you will want to set up alongside these defenses, see Multi-agent production patterns for the Claude API — it pairs naturally with this defensive architecture.

Where this architecture still falls short

Honest reporting requires acknowledging that even the full stack does not solve every problem. There are categories of error where this architecture helps less than you might hope, and recognizing them up front prevents misplaced confidence.

The first weakness is over-conservatism. Stack enough layers and Claude starts declining questions it could have answered correctly. In one deployment, declination rates climbed from 4% to 11% after I added the judge, and several of those declinations were against questions a human would have answered without hesitation. I only noticed the rise some time after adding the judge. The fix is to tune judge prompts to reward correctness, not just caution, and to monitor false-decline rates as carefully as false-fabrication rates.

The second weakness is shared error modes. If your retrieval pipeline returns the wrong documents, all the grounding in the world cannot help — Claude will faithfully ground on irrelevant text. Citations and judges become useless when the inputs themselves are off. This is why I treat retrieval quality as a first-class part of the defense rather than a separate concern. A weekly retrieval-quality eval, even on a small sample, catches drift before it shows up as user-visible errors.

Keep the eval narrow: do not run generation at all. Pull the retrieval layer out and count only whether an expected document landed in the top k. Write down 30 or so pairs of a question and its expected document IDs by hand, then run a script like this weekly.

// retrieval-eval.ts — measure the retrieval layer on its own (no generation calls)
type EvalCase = { question: string; expectedDocIds: string[] };
type Retrieve = (q: string, k: number) => Promise<{ id: string }[]>;
 
export async function evalRetrieval(cases: EvalCase[], retrieve: Retrieve, k = 5) {
  let hits = 0;
  let reciprocalSum = 0;
  const misses: string[] = [];
 
  for (const c of cases) {
    const ids = (await retrieve(c.question, k)).map((d) => d.id);
    const rank = ids.findIndex((id) => c.expectedDocIds.includes(id)); // 0-based; -1 when absent
    if (rank >= 0) {
      hits += 1;
      reciprocalSum += 1 / (rank + 1);
    } else {
      misses.push(c.question);
    }
  }
  return {
    hitRateAtK: hits / cases.length,   // share of cases where an expected doc landed in the top k
    mrr: reciprocalSum / cases.length, // how high the expected doc ranks (higher is better)
    misses,                            // the misses, for reading by eye
  };
}

What matters is not the absolute hitRateAtK but the change from last week. If it drops a few points after an index rebuild or an embedding-model swap, suspect retrieval before you touch any grounding settings. Read misses by eye instead of aggregating it — whether the cause is the phrasing of the question or a document that was never indexed decides where you fix it.

The third weakness is novel adversarial inputs. Prompt injection in user-supplied text can route around grounding, especially when the documents themselves contain attacker-controlled strings. Layer 6 schemas help reduce the surface area, and tool-use schemas with strict enums keep responses on rails, but you should still treat any system that ingests user content as exposed to injection regardless of these defenses. A separate review of injection-resistant prompt patterns is overdue in any production system.

Finally, this architecture does not address bias or harm beyond factuality. A perfectly grounded answer can still be culturally inappropriate, legally risky, or strategically misaligned with your brand voice. Those concerns require their own review layers — moderation, persona enforcement, content policy checks — that operate orthogonally to the hallucination defenses described here.

One step you can take tomorrow

If you only implement one layer from this article, start with Layer 2 (the escape-hatch tool). Adding the tool definitions to an existing system takes an afternoon, and you will feel the change in answer quality the same day. Just don't forget tool_choice: { type: "any" }.

From there, build the habit of measuring hallucination rates weekly. Add the next layer the moment your number crosses a threshold you cannot tolerate. That cadence keeps you from over-engineering and from under-protecting.

There is no perfect defense. You absorb the residual risk through non-technical layers too — clear caveats to the user, escalation paths to humans, and well-defined service boundaries. Claude API reaches production quality only when the engineering and the operational layers are designed together.