⬡ API & SDK/2026-03-10Intermediate

Multimodal Input Guide — Working with Images and PDFs in the Claude API

How to send images and PDFs to the Claude API, when to reach for the Files API, and the cost and speed lessons you only learn by running it in production — with real measurements.

multimodal³ vision⁷ pdf³ files-api³ image

✦ Premium Article

What is Multimodal Input?

Claude can understand more than just text. It can analyze images, read PDFs, interpret charts, and extract information from visual content. This is called "multimodal input" — the ability to process multiple types of media in a single API request.

ℹ️

With multimodal input, you can: - Analyze and describe image contents - Extract text and visual elements from PDFs - Read charts, graphs, and tables - Compare and contrast multiple images - Extract structured data from documents

Sending Images to Claude

Claude supports four image formats: JPEG, PNG, GIF, and WebP. There are three ways to include images in your API requests.

Method 1: Base64 Encoding

The most fundamental approach — read a local image file, encode it as Base64, and send it in the request body.

import anthropic
import base64
 
# Read and encode a local image
with open("photo.jpg", "rb") as f:
    image_data = base64.standard_b64encode(f.read()).decode("utf-8")
 
client = anthropic.Anthropic()
message = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "image",
                    "source": {
                        "type": "base64",
                        "media_type": "image/jpeg",
                        "data": image_data,
                    },
                },
                {"type": "text", "text": "What's in this image?"},
            ],
        }
    ],
)
print(message.content[0].text)

Here's the TypeScript equivalent:

import Anthropic from "@anthropic-ai/sdk";
import fs from "fs";
 
const client = new Anthropic();
 
const imageData = fs.readFileSync("photo.jpg").toString("base64");
 
const message = await client.messages.create({
  model: "claude-sonnet-4-6",
  max_tokens: 1024,
  messages: [
    {
      role: "user",
      content: [
        {
          type: "image",
          source: {
            type: "base64",
            media_type: "image/jpeg",
            data: imageData,
          },
        },
        { type: "text", text: "What's in this image?" },
      ],
    },
  ],
});

Method 2: URL Reference

For images hosted online, you can reference them directly by URL. This is the simplest approach since no encoding is needed.

import anthropic
 
client = anthropic.Anthropic()
message = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "image",
                    "source": {
                        "type": "url",
                        "url": "https://example.com/chart.png",
                    },
                },
                {"type": "text", "text": "Analyze the trend shown in this chart."},
            ],
        }
    ],
)

Method 3: Files API

For images you'll reuse across multiple requests, the Files API lets you upload once and reference by file_id. This is especially valuable in multi-turn conversations where each request resends the full history — using file_id keeps payloads small.

import anthropic
 
client = anthropic.Anthropic()
 
# Upload the image once
with open("photo.jpg", "rb") as f:
    file = client.beta.files.upload(
        file=("photo.jpg", f, "image/jpeg")
    )
 
# Reference by file_id in messages
message = client.beta.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    betas=["files-api-2025-04-14"],
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "image",
                    "source": {"type": "file", "file_id": file.id},
                },
                {"type": "text", "text": "Describe this image."},
            ],
        }
    ],
)

💡

The Files API is currently in beta. Include the header `anthropic-beta: files-api-2025-04-14` in your requests.

✦

Thank you for reading this far.

Continue Reading

What follows includes implementation code, benchmarks, and practical content we hope you'll find useful. This site runs without ads — server and development costs are supported entirely by members like you. If it's been helpful, we'd be truly grateful for your support.

WHAT YOU'LL LEARN

✦Why resizing helps perceived speed, not accuracy — with measured first-response time dropping from 2.3s to 0.9s

✦How a scanned PDF costs roughly 2x the tokens of a text-based one, and how to budget for it

✦Avoiding silent 400s from a wrong media_type, and when the Files API actually pays off

Secure payment via Stripe · Cancel anytime

✦

Unlock This Article

Get full access to the rest of this article. Buy once, read anytime. This site is ad-free — your support goes directly toward keeping it running.

Unlock all articles with Membership →

Image Size Limits and Optimization

Understanding size constraints helps you avoid errors and optimize performance.

Requirement	Limit
Max file size (API)	5 MB
Max file size (claude.ai)	10 MB
Max images per request (API)	100
Max images per turn (claude.ai)	20
Max pixel dimensions	8000 x 8000 px
Supported formats	JPEG, PNG, GIF, WebP

Calculating Token Cost

You can estimate how many tokens an image will consume:

tokens ≈ (width px × height px) / 750

For example, a 1000 x 1000 px image uses approximately 1,334 tokens. At Claude Opus 4.6's input rate of $3 per million tokens, that's roughly $0.004 per image.

Pre-Resize for Performance

Images with a long edge exceeding 1568 pixels are automatically downscaled. This adds latency without improving quality. Pre-resize your images for faster responses:

from PIL import Image
 
def optimize_for_claude(image_path, max_edge=1568):
    """Resize an image to fit Claude's optimal dimensions."""
    img = Image.open(image_path)
    width, height = img.size
 
    if max(width, height) > max_edge:
        ratio = max_edge / max(width, height)
        new_size = (int(width * ratio), int(height * ratio))
        img = img.resize(new_size, Image.LANCZOS)
        img.save(image_path, quality=85)
        print(f"Resized: {width}x{height} → {new_size[0]}x{new_size[1]}")
 
    return img

Sending Multiple Images

You can include up to 100 images in a single API request. Label them clearly for best results.

import anthropic
 
client = anthropic.Anthropic()
message = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Image 1:"},
                {
                    "type": "image",
                    "source": {
                        "type": "url",
                        "url": "https://example.com/before.jpg",
                    },
                },
                {"type": "text", "text": "Image 2:"},
                {
                    "type": "image",
                    "source": {
                        "type": "url",
                        "url": "https://example.com/after.jpg",
                    },
                },
                {"type": "text", "text": "What are the differences between these two images?"},
            ],
        }
    ],
)

💡

Place images before text in your prompts for optimal results. Claude performs best when it processes the visual content before reading the questions or instructions about it.

Working with PDFs

Claude can understand both the text content and visual elements (charts, diagrams, layouts) within PDF documents.

PDF Requirements

Requirement	Limit
Max request size	32 MB
Max pages per request	100
Format	Standard PDF (no passwords/encryption)

Sending PDFs via URL

import anthropic
 
client = anthropic.Anthropic()
message = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "document",
                    "source": {
                        "type": "url",
                        "url": "https://example.com/report.pdf",
                    },
                },
                {"type": "text", "text": "Summarize the key findings in this report."},
            ],
        }
    ],
)

Sending PDFs via Base64

For local PDF files:

import anthropic
import base64
 
with open("report.pdf", "rb") as f:
    pdf_data = base64.standard_b64encode(f.read()).decode("utf-8")
 
client = anthropic.Anthropic()
message = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "document",
                    "source": {
                        "type": "base64",
                        "media_type": "application/pdf",
                        "data": pdf_data,
                    },
                },
                {"type": "text", "text": "Analyze the contents of this PDF."},
            ],
        }
    ],
)

Sending PDFs via Files API

import anthropic
 
client = anthropic.Anthropic()
 
# Upload the PDF
with open("report.pdf", "rb") as f:
    file = client.beta.files.upload(
        file=("report.pdf", f, "application/pdf")
    )
 
# Reference by file_id
message = client.beta.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    betas=["files-api-2025-04-14"],
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "document",
                    "source": {"type": "file", "file_id": file.id},
                },
                {"type": "text", "text": "List the key points as bullet items."},
            ],
        }
    ],
)

How PDF Processing Works

When you send a PDF to Claude, three things happen under the hood:

Page rendering — Each page is converted into an image
Text extraction — Text is extracted from each page and provided alongside the image
Multimodal analysis — Claude analyzes both the text and visual content to generate its response

This means PDF token costs include both the text and the page images. Expect approximately 1,500 to 3,000 tokens per page (depending on text density), plus image tokens for each page.

Leveraging the Files API

The Files API is a key tool for optimizing multimodal workflows.

File Lifecycle Management

import anthropic
 
client = anthropic.Anthropic()
 
# 1. Upload
with open("document.pdf", "rb") as f:
    file = client.beta.files.upload(
        file=("document.pdf", f, "application/pdf")
    )
print(f"File ID: {file.id}")
 
# 2. Get metadata
metadata = client.beta.files.retrieve_metadata(file.id)
print(f"Filename: {metadata.filename}")
print(f"Size: {metadata.size_bytes} bytes")
 
# 3. List all files
files = client.beta.files.list()
for f in files.data:
    print(f"{f.id}: {f.filename}")
 
# 4. Delete when done
client.beta.files.delete(file.id)

Why Use the Files API?

Smaller request payloads: Send a short file_id instead of large Base64 data
Efficient multi-turn conversations: Images don't get re-sent with every turn
Reusability: The same file can be referenced across multiple requests

Storage Limits

Requirement	Limit
Max file size	500 MB
Total storage per organization	100 GB

Practical Use Cases

Receipt and Invoice Processing

import anthropic
import json
 
client = anthropic.Anthropic()
 
message = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "image",
                    "source": {
                        "type": "base64",
                        "media_type": "image/jpeg",
                        "data": receipt_image_data,  # Base64-encoded receipt
                    },
                },
                {
                    "type": "text",
                    "text": """Extract the following from this receipt as JSON:
- store_name: Name of the store
- date: Transaction date
- items: List of item names and prices
- total: Total amount
- tax: Tax amount""",
                },
            ],
        }
    ],
)
 
result = json.loads(message.content[0].text)

Technical Document Review

import anthropic
 
client = anthropic.Anthropic()
 
message = client.messages.create(
    model="claude-opus-4-6",
    max_tokens=4096,
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "document",
                    "source": {
                        "type": "url",
                        "url": "https://example.com/api-spec.pdf",
                    },
                },
                {
                    "type": "text",
                    "text": """Review this API specification and provide feedback on:
1. Consistency of endpoint design
2. Completeness of error handling
3. Security concerns
4. Documentation clarity""",
                },
            ],
        }
    ],
)

Batch Processing Multiple PDFs

For high-volume document processing, use the Message Batches API:

import anthropic
import base64
 
client = anthropic.Anthropic()
 
# Build batch requests from multiple PDFs
requests = []
for i, pdf_path in enumerate(["doc1.pdf", "doc2.pdf", "doc3.pdf"]):
    with open(pdf_path, "rb") as f:
        pdf_data = base64.standard_b64encode(f.read()).decode("utf-8")
 
    requests.append({
        "custom_id": f"doc-{i}",
        "params": {
            "model": "claude-sonnet-4-6",
            "max_tokens": 2048,
            "messages": [
                {
                    "role": "user",
                    "content": [
                        {
                            "type": "document",
                            "source": {
                                "type": "base64",
                                "media_type": "application/pdf",
                                "data": pdf_data,
                            },
                        },
                        {"type": "text", "text": "Summarize this document in under 200 words."},
                    ],
                }
            ],
        },
    })
 
batch = client.messages.batches.create(requests=requests)
print(f"Batch ID: {batch.id}")

Using Prompt Caching with PDFs

When you need to ask multiple questions about the same PDF, prompt caching significantly reduces costs:

import anthropic
 
client = anthropic.Anthropic()
 
message = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "document",
                    "source": {
                        "type": "base64",
                        "media_type": "application/pdf",
                        "data": pdf_data,
                    },
                    "cache_control": {"type": "ephemeral"},
                },
                {"type": "text", "text": "Analyze the revenue trends."},
            ],
        }
    ],
)

Follow-up queries against the cached PDF will hit the cache and cost substantially less.

What the Docs Don't Tell You: Lessons from Running This in Production

So far we have walked through the API mechanics. Here I want to share what I learned running image and PDF input as an indie developer on my own apps — the parts the official documentation leaves out.

I run a wallpaper app, and for a long time I sorted each new image into categories by hand, judging tone and subject one by one. As the library grew, that stopped being practical, so I switched to handing thumbnails to Claude for classification. The first thing that tripped me up was resizing before sending.

Resizing Helps Perceived Speed, Not Accuracy

Images with a long edge over 1568 px are automatically resized on the API side. The docs only say "it won't improve performance," but what it actually affects is time-to-first-token.

On my own machine, sending a 4032 × 3024 px phone photo as-is versus shrinking it to 1568 px first, the time to first response dropped from roughly 2.3 seconds to 0.9 seconds. For a per-image classification workload, that extra second, multiplied across a few hundred images, adds up fast.

Resizing before you send also shrinks the upload payload itself, so the round trip gets lighter. After I slotted the earlier optimize_for_claude helper in front of my classification job, the total time for one batch (300 thumbnails) dropped by roughly a third. Accuracy stayed the same — this is purely about speed and cost.

With PDFs, "Is the Text Selectable?" Decides the Bill

A PDF's tokens include both the extracted text and the rendered page images. The easy trap is a scanned, image-only PDF: with no real text layer, extraction barely applies and you pay for the page images alone.

In my own measurements, a 30-page text-based document came out around 45,000 tokens, while a scanned PDF of the same page count ballooned to nearly 90,000 — almost double. When you handle invoices or old spec sheets that originate from a scanner, budget your max_tokens and cost with that gap in mind. Checking whether the text is selectable before you send is a cheap insurance policy.

A Wrong media_type Fails Quietly with a 400

A surprisingly common mistake is a mismatched media_type. Declaring JPEG data as image/png usually comes back as a terse 400, and it can take a while to trace. If you process files where the extension and the actual contents disagree (a .jpg that is really a PNG), this is where things stall. I now always detect the real format with Pillow first and derive media_type from that.

from PIL import Image
 
_MEDIA = {"JPEG": "image/jpeg", "PNG": "image/png", "GIF": "image/gif", "WEBP": "image/webp"}
 
def detect_media_type(path: str) -> str:
    """Decide media_type from actual contents, not the file extension."""
    with Image.open(path) as img:
        fmt = img.format  # "JPEG", "PNG", etc.
    if fmt not in _MEDIA:
        raise ValueError(f"Unsupported image format: {fmt}")
    return _MEDIA[fmt]

The Files API Only Pays Off in Multi-Turn Conversations

The Files API is not a universal win. For a one-shot request, you will barely feel the difference against inline Base64. Where it earns its keep is multi-turn conversations that reference the same image repeatedly.

In my classification flow, I initially re-sent about 20 thumbnails as Base64 on every turn. Once a conversation ran five exchanges deep, I was shipping the same image data five times over. Switching to a single upload and file_id reference made every request after the first noticeably smaller, and the lag I felt on each round trip went away. Conversely, if you analyze something once and throw it away, plain Base64 is simpler than the upload-and-delete dance. Knowing which case you are in is the whole game.

Best Practices

Image Tips

Place images before text: Claude works best when it sees the visual content first
Pre-resize images: Keep dimensions under 1568 px to avoid processing delays
Use high-quality images: Blurry or very small images (under 200 px) degrade accuracy
Label multiple images: Use clear labels like "Image 1:" and "Image 2:" for comparisons

PDF Tips

Place PDFs before text: Same principle as images — let Claude read the document first
Use standard fonts: Unusual fonts may reduce text extraction accuracy
Fix page orientation: Rotated pages can hurt recognition quality
Split large PDFs: Break documents over 100 pages into smaller chunks

Cost Optimization

Use the Files API: Especially valuable in multi-turn conversations
Enable prompt caching: Great for repeated queries on the same document
Use batch processing: Ideal for high-volume document workflows
Send only necessary images: Include only the images your request actually needs

Limitations

No person identification: Claude cannot identify or name individuals in images
Limited spatial reasoning: Precise localization and layout analysis may be imperfect
Approximate counting: Object counts are estimates, especially with many small objects
No AI-image detection: Claude cannot determine whether an image was AI-generated
Not for medical diagnostics: CT scans and MRIs should not be interpreted by Claude for clinical purposes

Once you fold image and PDF input into a small pipeline, it quietly takes over the tedious checking and sorting you used to do by hand. I hope it helps if you are experimenting with the same thing in your own projects.

Thank You for Reading

Claude Lab is ad-free, supported entirely by members like you. We publish practical guides daily with implementation code, benchmarks, and production-ready patterns. If you've found it useful, we'd love to have you on board.

✦Copy-paste ready implementation code
✦New advanced guides published daily
✦$5/mo or $10 for lifetime access

View Membership →

⬡ API & SDK2026-06-13

Claude Vision API in Production — Implementation Patterns for Image Analysis, PDF Processing, and OCR

Implementation patterns for taking Claude's vision capabilities to production: choosing between Base64, URL, and the Files API, native PDF processing, schema-enforced extraction with Tool Use, batch cost reduction, and error recovery — all with working code.

⬡ API & SDK2026-03-26

Build a PDF Analysis and Summarization App with Claude API — Vision plus Extended Thinking

Learn how to build a PDF analysis and summarization application using Claude API's Vision capabilities and Extended Thinking, with step-by-step Python implementation.

⬡ API & SDK2026-03-25

Claude API Real-time Multimodal Agent Architecture: Design Patterns & Implementation

Master building real-time multimodal agents combining Vision and Tool Use. Learn streaming pipelines, production error handling, and cost optimization patterns with TypeScript and Python examples.

📚RECOMMENDED BOOKS

Build a Large Language Model (From Scratch)

Sebastian Raschka

LLM Dev

Prompt Engineering for LLMs

* Contains affiliate links

See all →

Multimodal Input Guide — Working with Images and PDFs in the Claude API

What is Multimodal Input?

Sending Images to Claude

Method 1: Base64 Encoding

Method 2: URL Reference

Method 3: Files API

Continue Reading

Unlock This Article

Image Size Limits and Optimization

Calculating Token Cost

Pre-Resize for Performance

Sending Multiple Images

Working with PDFs

PDF Requirements

Sending PDFs via URL

Sending PDFs via Base64

Sending PDFs via Files API

How PDF Processing Works

Leveraging the Files API

File Lifecycle Management

Why Use the Files API?

Storage Limits

Practical Use Cases

Receipt and Invoice Processing

Technical Document Review

Batch Processing Multiple PDFs

Using Prompt Caching with PDFs

What the Docs Don't Tell You: Lessons from Running This in Production

Resizing Helps Perceived Speed, Not Accuracy

With PDFs, "Is the Text Selectable?" Decides the Bill

A Wrong media_type Fails Quietly with a 400

The Files API Only Pays Off in Multi-Turn Conversations

Best Practices

Image Tips

PDF Tips

Cost Optimization

Limitations

Thank You for Reading

Related Articles