⬡ API & SDK/2026-05-05Advanced

Building a 'Think-and-Search' AI Agent — Claude API Extended Thinking × Tool Use

A deep dive into combining Claude API Extended Thinking and Tool Use. Covers frequent errors, a complete research agent implementation in Python, plus cost estimation, timeout design, and error recovery for production use.

extended-thinking⁷ tool-use²² claude-api⁸¹ python²² agent¹³ api-sdk¹³

✦ Premium Article

When Claude Sonnet 4.6 launched, it brought something I had been waiting for: Extended Thinking and Tool Use working together in the same API call. Before this, you had to choose — either let Claude think deeply, or let it call external tools. Not both. With that limitation removed, it became possible to build agents that genuinely reason about what to search, plan a sequence of tool calls, and then interpret the results — all in one coherent loop.

Getting there took a frustrating couple of hours, though. The max_tokens setting kept tripping me up, I had a wrong mental model of what budget_tokens actually means, and handling thinking_delta events during streaming was completely undocumented in the examples I could find. This guide is the resource I wish had existed when I started.

The focus here is working code and the reasoning behind each design choice, not theory. I'm assuming you have already made basic Anthropic API calls and want to go further.

What Becomes Possible When You Combine Both

A quick framing before the code.

Extended Thinking lets Claude run a long internal reasoning process before returning an answer. That reasoning appears in the response as thinking blocks. It excels at multi-step problems where getting to the right answer requires working through intermediate conclusions.

Tool Use lets Claude call external functions during a conversation — web search, database queries, calculations, file reads. Claude signals which tool to call (and with what arguments), your code runs it, and you feed the result back.

Together, the flow looks like this:

Receive user task
Extended Thinking: plan which tools to call, and in what order
Tool Use: execute the planned tools
Extended Thinking: interpret results, decide whether to continue or conclude
Repeat steps 3–4 if needed
Generate final answer

This is qualitatively different from simple tool use. Claude is not just pattern-matching "user asked about prices, call price_lookup tool." It is reasoning about strategy, noticing when a result is incomplete, and adapting its approach mid-task.

Constraints to Understand Before Writing Any Code

The combination has specific constraints that will cause errors if you ignore them. These are worth learning upfront.

Supported Models

Extended Thinking with Tool Use requires one of:

claude-opus-4-6 — highest reasoning quality, highest cost
claude-sonnet-4-6 — strong balance of quality and cost, the practical default

claude-haiku-4-5 does not support Extended Thinking. For prototypes and development, start with claude-sonnet-4-6 and upgrade to Opus only if you need deeper reasoning on genuinely complex tasks.

The max_tokens Trap

This is where most people first hit an error.

When Extended Thinking is active, max_tokens must accommodate both the thinking tokens and the output tokens. If max_tokens is smaller than budget_tokens, the API rejects the request.

# This will error
response = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,           # too small
    thinking={
        "type": "enabled",
        "budget_tokens": 5000  # budget_tokens > max_tokens → error
    },
    tools=[...],
    messages=[...]
)
# Error: thinking budget_tokens must be less than max_tokens
 
# Correct pattern
response = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=16000,           # budget_tokens + room for output
    thinking={
        "type": "enabled",
        "budget_tokens": 10000  # stays within max_tokens
    },
    tools=[...],
    messages=[...]
)

budget_tokens is the ceiling on thinking tokens Claude may use — not a guarantee it will use that many. Check response.usage to see actual consumption and tune from there.

tool_choice Recommendation

Use {"type": "auto"} rather than {"type": "any"}. The any option forces a tool call regardless of whether one is needed. When Extended Thinking is active, you want Claude to reason freely and potentially decide no tool is necessary. Forcing tool use interferes with that judgment.

✦

Thank you for reading this far.

Continue Reading

What follows includes implementation code, benchmarks, and practical content we hope you'll find useful. This site runs without ads — server and development costs are supported entirely by members like you. If it's been helpful, we'd be truly grateful for your support.

WHAT YOU'LL LEARN

✦Measured data showing thinking-token usage plateaus even as you raise budget_tokens

✦Concrete fixes for history-management pitfalls, including preserving thinking blocks after stop_reason=tool_use

✦Complete runnable code for both research and code-review agents, plus production cost and timeout design

Secure payment via Stripe · Cancel anytime

✦

Unlock This Article

Get full access to the rest of this article. Buy once, read anytime. This site is ad-free — your support goes directly toward keeping it running.

Unlock all articles with Membership →

Minimum Working Implementation

Before building the full agent, verify the combination actually works with this stripped-down example.

import anthropic
import json
 
client = anthropic.Anthropic()
 
tools = [
    {
        "name": "calculate",
        "description": "Evaluate a mathematical expression and return the result.",
        "input_schema": {
            "type": "object",
            "properties": {
                "expression": {
                    "type": "string",
                    "description": "Python math expression (e.g. '2 ** 32 + 100')"
                }
            },
            "required": ["expression"]
        }
    }
]
 
def handle_tool_call(tool_name: str, tool_input: dict) -> str:
    if tool_name == "calculate":
        try:
            result = eval(tool_input["expression"], {"__builtins__": {}}, {})
            return str(result)
        except Exception as e:
            return f"Calculation error: {e}"
    return "Unknown tool"
 
def run_agent(user_message: str) -> str:
    messages = [{"role": "user", "content": user_message}]
    max_iterations = 5
 
    for iteration in range(max_iterations):
        response = client.messages.create(
            model="claude-sonnet-4-6",
            max_tokens=16000,
            thinking={
                "type": "enabled",
                "budget_tokens": 8000
            },
            tools=tools,
            tool_choice={"type": "auto"},
            messages=messages
        )
 
        print(f"[Iteration {iteration + 1}] stop_reason: {response.stop_reason}")
 
        for block in response.content:
            if block.type == "thinking":
                print(f"  [thinking] {block.thinking[:200]}...")
 
        if response.stop_reason == "end_turn":
            for block in response.content:
                if block.type == "text":
                    return block.text
            return ""
 
        if response.stop_reason == "tool_use":
            # Pass the full response.content — thinking blocks included
            messages.append({"role": "assistant", "content": response.content})
 
            tool_results = []
            for block in response.content:
                if block.type == "tool_use":
                    print(f"  [tool call] {block.name}({block.input})")
                    result = handle_tool_call(block.name, block.input)
                    print(f"  [tool result] {result}")
                    tool_results.append({
                        "type": "tool_result",
                        "tool_use_id": block.id,
                        "content": result
                    })
 
            messages.append({"role": "user", "content": tool_results})
 
    return "Reached maximum iterations"
 
result = run_agent(
    "Calculate 2 to the power of 32 plus 100, then find the square root of that result. "
    "Walk me through each step."
)
print("\n=== Final Answer ===")
print(result)

The most important detail: when appending the assistant turn after a tool call, pass response.content as-is — the full list including thinking blocks. If you extract only text blocks, Claude loses the reasoning context that motivated the tool call. This causes subtle quality degradation on subsequent iterations and can trigger parsing errors.

Complete Research Agent Implementation

The full implementation uses three tools — web search, file reading, and calculation — with Extended Thinking guiding the strategy across multiple iterations.

import anthropic
import json
from pathlib import Path
 
client = anthropic.Anthropic()
 
TOOLS = [
    {
        "name": "web_search",
        "description": (
            "Search the web for current information on a topic. "
            "Use when facts may be recent or uncertain."
        ),
        "input_schema": {
            "type": "object",
            "properties": {
                "query": {"type": "string", "description": "Search query string"},
                "max_results": {
                    "type": "integer",
                    "description": "Maximum results to return (default 5)",
                    "default": 5
                }
            },
            "required": ["query"]
        }
    },
    {
        "name": "read_file",
        "description": "Read the contents of a local text file.",
        "input_schema": {
            "type": "object",
            "properties": {
                "file_path": {"type": "string", "description": "Path to the file"}
            },
            "required": ["file_path"]
        }
    },
    {
        "name": "calculate",
        "description": "Evaluate a Python math expression. Supports math module functions.",
        "input_schema": {
            "type": "object",
            "properties": {
                "expression": {"type": "string", "description": "Expression to evaluate"}
            },
            "required": ["expression"]
        }
    }
]
 
def execute_tool(tool_name: str, tool_input: dict) -> str:
    """Execute a tool and return the result as a string. Never raises — errors become strings."""
    try:
        if tool_name == "web_search":
            # Replace with SerpAPI / Tavily in production:
            # import requests, os
            # resp = requests.get("https://serpapi.com/search", params={
            #     "q": tool_input["query"],
            #     "api_key": os.getenv("SERPAPI_KEY"),
            #     "num": tool_input.get("max_results", 5)
            # })
            # results = resp.json().get("organic_results", [])
            # return "\n".join(f"- {r['title']}: {r['snippet']}" for r in results)
            return f"[mock] Search results for: {tool_input['query']} (replace with SerpAPI/Tavily)"
 
        elif tool_name == "read_file":
            path = Path(tool_input["file_path"])
            if not path.exists():
                return f"Error: file not found: {path}"
            if path.stat().st_size > 100_000:
                return "Error: file exceeds 100KB limit"
            return path.read_text(encoding="utf-8")
 
        elif tool_name == "calculate":
            import math
            safe_env = {
                "abs": abs, "round": round, "min": min, "max": max,
                "sum": sum, "int": int, "float": float,
                **vars(math)
            }
            result = eval(tool_input["expression"], {"__builtins__": {}}, safe_env)
            return str(result)
 
        else:
            return f"Error: unknown tool: {tool_name}"
 
    except Exception as e:
        return f"Tool error ({tool_name}): {type(e).__name__}: {e}"
 
class ResearchAgent:
    def __init__(
        self,
        model: str = "claude-sonnet-4-6",
        budget_tokens: int = 10000,
        max_tokens: int = 16000,
        max_iterations: int = 8,
        verbose: bool = True
    ):
        self.model = model
        self.budget_tokens = budget_tokens
        self.max_tokens = max_tokens
        self.max_iterations = max_iterations
        self.verbose = verbose
 
    def run(self, task: str, context: str = "") -> dict:
        """
        Run a task and return the result.
 
        Returns a dict with:
            answer (str): final answer text
            thinking_steps (list): per-iteration thinking content
            tool_calls (list): record of tool calls made
            total_usage (dict): token consumption
        """
        system = (
            "You are a thorough research agent. Use the available tools to complete tasks accurately.\n"
            "When information may be outdated or uncertain, search before answering.\n"
            "Use calculate for any numeric computations — do not compute in your head."
        )
        if context:
            system += f"\n\nContext:\n{context}"
 
        messages = [{"role": "user", "content": task}]
        thinking_steps, tool_calls = [], []
        input_tokens = output_tokens = 0
 
        for iteration in range(self.max_iterations):
            if self.verbose:
                print(f"\n{'='*50}\nIteration {iteration + 1}/{self.max_iterations}")
 
            response = client.messages.create(
                model=self.model,
                max_tokens=self.max_tokens,
                thinking={"type": "enabled", "budget_tokens": self.budget_tokens},
                tools=TOOLS,
                tool_choice={"type": "auto"},
                system=system,
                messages=messages
            )
 
            input_tokens += response.usage.input_tokens
            output_tokens += response.usage.output_tokens
 
            for block in response.content:
                if block.type == "thinking":
                    thinking_steps.append({"iteration": iteration + 1, "thinking": block.thinking})
                    if self.verbose:
                        print(f"[thinking] {block.thinking[:300].replace(chr(10), ' ')}...")
 
            if response.stop_reason == "end_turn":
                answer = "".join(b.text for b in response.content if b.type == "text")
                if self.verbose:
                    print(f"\n[done] Finished in {iteration + 1} iterations")
                    print(f"[usage] in={input_tokens} out={output_tokens} tokens")
                return {
                    "answer": answer,
                    "thinking_steps": thinking_steps,
                    "tool_calls": tool_calls,
                    "total_usage": {"input_tokens": input_tokens, "output_tokens": output_tokens}
                }
 
            if response.stop_reason == "tool_use":
                # Append full content — thinking blocks must be included
                messages.append({"role": "assistant", "content": response.content})
 
                tool_results = []
                for block in response.content:
                    if block.type == "tool_use":
                        if self.verbose:
                            print(f"[tool] {block.name}({json.dumps(block.input)})")
 
                        result = execute_tool(block.name, block.input)
 
                        # Trim oversized results to prevent context overflow
                        if len(result) > 5000:
                            result = result[:5000] + "\n\n[truncated — first 5000 chars shown]"
 
                        if self.verbose:
                            print(f"  → {result[:150].replace(chr(10), ' ')}...")
 
                        tool_calls.append({
                            "iteration": iteration + 1,
                            "tool": block.name,
                            "input": block.input,
                            "result_preview": result[:200]
                        })
                        tool_results.append({
                            "type": "tool_result",
                            "tool_use_id": block.id,
                            "content": result
                        })
 
                messages.append({"role": "user", "content": tool_results})
 
        return {
            "answer": "Reached max iterations. Try splitting the task or increasing budget_tokens.",
            "thinking_steps": thinking_steps,
            "tool_calls": tool_calls,
            "total_usage": {"input_tokens": input_tokens, "output_tokens": output_tokens}
        }
 
if __name__ == "__main__":
    agent = ResearchAgent(
        model="claude-sonnet-4-6",
        budget_tokens=10000,
        max_tokens=16000,
        max_iterations=8,
        verbose=True
    )
 
    result = agent.run(
        task=(
            "Research the major model updates Anthropic announced in 2026, "
            "then compare the pricing of claude-sonnet-4-6 and claude-haiku-4-5. "
            "Calculate the cost difference for 10 million input tokens and 1 million output tokens."
        )
    )
 
    print("\n" + "="*50)
    print("Final answer:")
    print(result["answer"])

Three design decisions worth calling out explicitly:

Result length limits (5000 chars): Unbounded tool results eat your context window fast. Without trimming, long tasks reliably hit context_window_exceeded. The 5000-char limit is conservative — adjust based on your typical tool output sizes.

Full response.content in message history: When appending the assistant turn, pass the entire response.content list — thinking, text, and tool_use blocks all together. Extracting only text blocks strips the reasoning context that connects the thinking to the tool call. Subsequent iterations lose coherence.

Tools never raise exceptions: execute_tool() catches all exceptions and returns them as strings. Claude reads the error string and decides how to recover — try a different query, use a different tool, or tell the user it could not complete the task. If exceptions propagate, the agent loop dies.

Common Pitfalls and Fixes

Problems I actually ran into during implementation.

Pitfall 1: Empty thinking blocks

When budget_tokens is too low, Claude decides there is not enough room to think and returns an empty thinking block. The call succeeds but reasoning quality drops significantly.

Fix: Start at budget_tokens=5000 for simple tasks, 8000–12000 for multi-step research. Check response.usage to see actual thinking token consumption and tune from there.

Pitfall 2: Context bloat in long sessions

In multi-turn conversations, thinking blocks from past iterations accumulate in message history. After several rounds, the context can grow large enough to cause slowness or overflow errors.

Fix: Compress past assistant messages by removing their thinking blocks:

def compress_messages(messages: list) -> list:
    """Remove thinking blocks from all but the most recent assistant turn."""
    if len(messages) < 4:
        return messages
    compressed = []
    for i, msg in enumerate(messages):
        if msg["role"] == "assistant" and i < len(messages) - 2:
            content = [
                b for b in msg["content"]
                if not (hasattr(b, "type") and b.type == "thinking")
            ]
            compressed.append({"role": "assistant", "content": content})
        else:
            compressed.append(msg)
    return compressed

Pitfall 3: Streaming event handling

In streaming mode, thinking_delta events appear alongside text deltas. If your parser only handles text_delta, you will get an unexpected event type.

with client.messages.stream(
    model="claude-sonnet-4-6",
    max_tokens=16000,
    thinking={"type": "enabled", "budget_tokens": 8000},
    tools=tools,
    messages=messages
) as stream:
    current_block_type = None
 
    for event in stream:
        if hasattr(event, "type"):
            if event.type == "content_block_start":
                if hasattr(event.content_block, "type"):
                    current_block_type = event.content_block.type
 
            elif event.type == "content_block_delta":
                if current_block_type == "text" and hasattr(event.delta, "text"):
                    print(event.delta.text, end="", flush=True)
                # thinking_delta: ignore or collect separately

Production Considerations — Cost, Latency, and Model Choice

Cost

Thinking tokens count as output tokens for billing purposes. At claude-sonnet-4-6 rates (as of May 2026):

Input: $3 / million tokens
Output: $15 / million tokens

With budget_tokens=10000, a single request can cost up to $0.15 in output tokens alone. An 8-iteration agent run could reach $1.20 in output costs before the final answer is generated.

In practice, Claude rarely uses the full budget_tokens allocation, but plan conservatively. During development, set budget_tokens=3000 to keep costs down until your logic is validated.

Latency

Extended Thinking adds significant latency compared to standard calls:

budget_tokens=5000: ~15–30 seconds
budget_tokens=10000: ~30–60 seconds
budget_tokens=15000: ~60–90 seconds

For web-facing endpoints, use streaming so users see progress rather than a blank wait. For batch tasks, consider running agents as background jobs with polling for results.

Sonnet vs Opus

From practical experience: claude-sonnet-4-6 handles the majority of research and analysis tasks well at budget_tokens=8000–12000. Opus produces noticeably deeper reasoning but costs 3–5x more and is meaningfully slower.

Opus is worth the premium for tasks where getting the wrong answer is expensive — legal document analysis, complex financial modeling, scientific reasoning. For typical development and research workflows, Sonnet is the right default.

Applying This to Code Review

Beyond research, code review is one of the strongest use cases for this combination. Extended Thinking lets Claude reason about why something is a problem and how it could be exploited, not just flag surface patterns.

def code_review_agent(file_path: str, criteria: list) -> dict:
    """
    Read a file and produce a structured code review using Extended Thinking.
 
    criteria examples:
        ["SQL injection and auth bypass vulnerabilities",
         "N+1 query and loop performance issues",
         "Unhandled exceptions and swallowed errors"]
    """
    agent = ResearchAgent(
        model="claude-sonnet-4-6",
        budget_tokens=12000,  # code review benefits from deeper thinking
        max_tokens=18000,
        max_iterations=3,
        verbose=False
    )
 
    criteria_text = "\n".join(f"- {c}" for c in criteria)
 
    task = (
        f"Code review request for: {file_path}\n\n"
        f"Review dimensions:\n{criteria_text}\n\n"
        f"Use read_file to load the file first, then provide specific feedback "
        f"for each dimension with line numbers and code excerpts. "
        f"If no issues found in a dimension, say so explicitly."
    )
 
    return agent.run(task)
 
result = code_review_agent(
    file_path="./src/api/checkout.py",
    criteria=[
        "SQL injection and authentication bypass vulnerabilities",
        "Performance issues including N+1 queries and unnecessary loops",
        "Missing error handling and silently swallowed exceptions"
    ]
)
print(result["answer"])

What the Docs Don't Tell You — Lessons From Running This in Production

Below are the things I only learned after wiring this agent into a real backend and running it for a few days. None of it is in the official docs, but all of it matters once you go live.

Thinking Blocks Get Thrown Away on Every Retry

An easy one to miss: after stop_reason comes back as tool_use, you have to append the assistant message back into history exactly as received. If you strip the thinking block and keep only the tool_use block, the next call fails with a validation error complaining that the thinking block is missing. Treat the thinking and the tool call as one indivisible turn — that framing keeps you out of trouble.

For handling stop_reason more broadly, I wrote up the patterns separately in the stop_reason handling guide.

Numbers I Measured in My Own Environment

I run an indie wallpaper app, and I wired this pattern into its support-inquiry classification backend (a few hundred requests a day). These are the numbers I measured with claude-sonnet-4-6 — all medians over 100 requests.

budget_tokens cap	Actual thinking tokens used (median)	Latency (median)	Output cost per request
3,000	~1,900	8.2 s	$0.032
8,000	~2,100	11.6 s	$0.038
12,000	~2,050	12.1 s	$0.037

What surprised me was that raising the budget_tokens cap does not raise the thinking tokens actually consumed. For a relatively simple task like classification, actual usage plateaued around 2,000 even with a cap of 12,000. Pushing the cap higher just added latency and cost for nothing.

Tune the cap to the complexity of the task. That was the single most effective adjustment in production: start at 3,000, and only step it up when you see signs of thinking being cut short — shallow conclusions, reasoning that stops mid-way.

Under Load, "Re-Thinking" Is What Drives Cost

The other thing that bites you is behavior under load. When a 529 Overloaded triggers a retry, that request re-runs its thinking from scratch. The thinking tokens on the failed attempt aren't billed, but the latency until success accumulates the full thinking time each attempt.

So I went with a two-tier setup: latency-sensitive endpoints like classification pin budget_tokens low (around 3,000), while only the asynchronous jobs that genuinely need deep reasoning get a higher cap. For the specifics of handling overload, I separated that out into the 529 Overloaded error guide.

Where to Go From Here

The fastest path to getting this running in your own project:

Start with the minimum working implementation — just copy it, run it, confirm Extended Thinking and Tool Use are both active. That takes about 30 minutes and eliminates all the configuration guesswork.

Once that works, replace the mock web_search implementation with SerpAPI or Tavily. That is another 1–2 hours and gives you a genuinely useful research tool.

From there, add tools specific to your domain. File readers, database queries, API calls to your own services — the agent loop handles all of them the same way.

For deeper coverage of Tool Use design patterns, see Claude API Advanced Tool Use Guide. For cost optimization strategies across your broader API usage, Anthropic API Cost Optimization Guide covers prompt caching and batching approaches that pair well with extended thinking workflows.

Thank You for Reading

Claude Lab is ad-free, supported entirely by members like you. We publish practical guides daily with implementation code, benchmarks, and production-ready patterns. If you've found it useful, we'd love to have you on board.