After more than a decade of indie app development, you stop noticing friction as a number and start noticing it as fatigue. For my wallpaper apps, the fatigue lived in image categorization.
Every time I added a new batch of wallpapers, I'd manually tag each one with a category: nature, cityscape, abstract, animals, and so on. At 10–15 seconds per image, 500 new wallpapers meant roughly two hours of repetitive clicking. With multiple updates per month, this adds up fast.
When Claude Vision API became available, I saw an opportunity. This is an honest account of what happened when I tried to automate that categorization — what worked, what didn't, and what the numbers actually looked like.
Why Existing Image Classification Wasn't Enough
This wasn't my first attempt at automation. I had tried Google Cloud Vision API and Apple's Vision framework before. The problem wasn't technical accuracy — it was the type of accuracy that mattered.
Conventional image classifiers are good at objective labels: "dog," "building," "sunset." But a wallpaper app needs categories that match how users feel when browsing. A photo of a sunset cityscape might get labeled "sky, building, dusk" by Google Vision — technically correct, but I'd want it in the "Evening Glow & Dusk" category, which is what users actually search for.
Claude's language model foundation means you can say "classify this according to our category system" and have it actually understand what you mean. That was the hypothesis worth testing.
Implementation: The Core Classification Script
Here's the simplified version of what runs in production:
import anthropic
import base64
import json
from pathlib import Path
client = anthropic.Anthropic(api_key="YOUR_API_KEY")
# Our app's category system
CATEGORIES = [
"Nature & Scenery", "City & Architecture", "Abstract & Pattern",
"Animals & Wildlife", "Flowers & Plants", "Ocean & Waterside",
"Night Sky & Dusk", "Space & Cosmos", "Seasons & Events",
"Minimal & Simple"
]
def classify_wallpaper(image_path: str) -> dict:
"""Classify a single wallpaper image."""
with open(image_path, "rb") as f:
image_data = base64.standard_b64encode(f.read()).decode("utf-8")
ext = Path(image_path).suffix.lower()
media_type_map = {".jpg": "image/jpeg", ".jpeg": "image/jpeg",
".png": "image/png", ".webp": "image/webp"}
media_type = media_type_map.get(ext, "image/jpeg")
response = client.messages.create(
model="claude-haiku-4-5-20251001", # Cost-optimized with Haiku
max_tokens=256,
messages=[{
"role": "user",
"content": [
{
"type": "image",
"source": {
"type": "base64",
"media_type": media_type,
"data": image_data
}
},
{
"type": "text",
"text": f"""This image is for a wallpaper app. Choose the most appropriate category from the list below and respond in JSON format only.
Categories: {json.dumps(CATEGORIES)}
Response format:
{{"category": "category name", "confidence": 0.0-1.0, "reason": "one-sentence explanation"}}"""
}
]
}]
)
result_text = response.content[0].text.strip()
return json.loads(result_text)The First Bug Wasn't in the API — It Was in File Discovery
Before any of that ran, I needed a list of images. My first attempt looked reasonable:
image_files = list(Path(image_dir).glob("*.{jpg,jpeg,png,webp}"))If you write shell for a living, that reads as "four extensions." It returns nothing. pathlib.Path.glob() does not perform brace expansion — it treats {jpg,jpeg,png,webp} as literal characters.
Verifying it takes ten seconds:
from pathlib import Path
# In a directory containing a.jpg, b.png, c.webp, d.jpeg
p = Path(".")
print(list(p.glob("*.{jpg,jpeg,png,webp}"))) # -> []
print([x.name for x in p.iterdir()
if x.suffix.lower() in {".jpg", ".jpeg", ".png", ".webp"}])
# -> ['a.jpg', 'b.png', 'c.webp', 'd.jpeg']What made this expensive was the silence. No exception, no warning — the script ran cleanly and printed Done: 0 items saved. I spent half an hour suspecting my API key and model name before realizing execution never reached the loop.
Here is the version I actually use, plus the three lines that would have saved me that half hour:
IMAGE_EXTS = {".jpg", ".jpeg", ".png", ".webp"}
def collect_images(image_dir: str) -> list:
"""Collect image files from a directory, sorted for stable ordering."""
return sorted(
p for p in Path(image_dir).iterdir()
if p.is_file() and p.suffix.lower() in IMAGE_EXTS
)
image_files = collect_images(image_dir)
if not image_files:
raise SystemExit(f"No images found in: {image_dir}")
print(f"Target: {len(image_files)} images")Cheap insurance. In any API-driven batch job, confusing "empty input" with "failed processing" is the most expensive mistake you can make, because neither one leaves a stack trace.
Haiku vs. Sonnet: The Cost-Accuracy Tradeoff
This was a real decision point. I ran 100 images through each model and compared accuracy against my own manual labels.
Claude Haiku 4.5: 89% category match, ~$0.50–$1.00 per 500 images Claude Sonnet 4.5: 94% category match, ~$3.00–$6.00 per 500 images
The 5% accuracy gap between models is real, but I looked at what Haiku got wrong. Almost all of the errors were borderline cases — images that could reasonably belong to two categories. A forest path at dusk could be "Nature" or "Evening Glow." A wave crashing on rocks could be "Ocean" or "Nature." These aren't wrong enough to mislead users.
For a task where human-level consistency is the standard (not medical-grade precision), Haiku at roughly 1/5 the cost was the right call. For 500 images per month, that means $6–10/year versus $36–72/year. Not life-changing, but it adds up over multiple apps.
Improving Claude Vision accuracy through image preprocessing has useful techniques that apply here too, especially around resolution and cropping.
Real Accuracy: Manual Verification of 75 Images
After processing 500 wallpapers, I pulled a random sample of 75 and labeled them myself to check Claude's work.
Results:
- Exact match: 66 images (88%)
- Acceptable near-match (e.g., Night Sky vs. City): 7 images (9.3%)
- Clear error: 2 images (2.7%)
The two clear errors:
- A forest path — classified as "Abstract & Pattern" (should be "Nature & Scenery")
- A silhouette figure against a space background — classified as "Space & Cosmos" (closer to "Abstract & Pattern")
Interestingly, both were images I'd have found ambiguous myself. The 88% exact match rate is better than I expected, and 2.7% outright errors is comparable to the mistake rate I make when doing 300+ images in a single session while tired.
Three Production Pitfalls (and How I Fixed Them)
1. JSON parsing failures happen more than you'd expect
Even with explicit JSON-only instructions, Claude occasionally adds explanatory text. Build in a safety net:
import re
def safe_parse_json(text: str) -> dict:
"""Extract JSON from a response that might have extra text."""
# Try code block first
code_match = re.search(r'```(?:json)?\n(.*?)\n```', text, re.DOTALL)
if code_match:
return json.loads(code_match.group(1))
# Fall back to raw JSON object
json_match = re.search(r'\{.*\}', text, re.DOTALL)
if json_match:
return json.loads(json_match.group())
raise ValueError(f"No JSON found in: {text[:100]}")If you need the response shape locked down hard, the layered approach in bulletproof JSON responses with prefill is worth the setup. For classification this lightweight, prompt-based JSON plus a fallback parser was enough.
2. Rate limits will catch you
500 images in one burst will trigger rate limits depending on your tier. Always control request pacing:
import time
def batch_classify_with_rate_limit(image_files: list, max_per_minute: int = 40):
results = {}
interval = 60.0 / max_per_minute
for i, img_path in enumerate(image_files):
start = time.time()
result = classify_wallpaper(str(img_path))
results[str(img_path)] = result
elapsed = time.time() - start
sleep_time = max(0, interval - elapsed)
if sleep_time > 0:
time.sleep(sleep_time)
if (i + 1) % 50 == 0:
print(f" → Processed {i+1} images")
return resultsSee API Rate Limits Best Practices for the full picture on handling 429 errors and backoff strategies.
3. High-resolution images slow everything down
Wallpaper images are often 3,000×5,000px or larger. Base64-encoding these means megabytes of data per request. Processing 500 images at full resolution took nearly three times as long as necessary.
For categorization purposes, a 512px thumbnail is sufficient. Perception-level category judgment doesn't need full resolution:
from PIL import Image
import io
def resize_for_classification(image_path: str, max_size: int = 512) -> bytes:
"""Resize image for classification while preserving aspect ratio."""
with Image.open(image_path) as img:
img.thumbnail((max_size, max_size), Image.LANCZOS)
buffer = io.BytesIO()
img.save(buffer, format="JPEG", quality=85)
return buffer.getvalue()After adding this step, total processing time for 500 images dropped from ~45 minutes to ~15 minutes.
Assume the Job Will Die Halfway Through
Pacing 500 requests to stay under rate limits means the job runs for fifteen minutes or more. In that window, networks drop and terminals get closed. For a while I restarted from the top every time, paying twice for images I had already classified.
The fix is unglamorous: read the results file before you start.
def load_done(output_file: str) -> dict:
"""Load prior results, if any."""
path = Path(output_file)
if not path.exists():
return {}
with path.open(encoding="utf-8") as f:
done = json.load(f)
# Retry anything that previously errored
return {k: v for k, v in done.items() if "error" not in v}
def batch_classify_resumable(image_dir: str, output_file: str, save_every: int = 20):
"""Batch classification that can resume after an interruption."""
results = load_done(output_file)
targets = [p for p in collect_images(image_dir) if p.name not in results]
print(f"Target: {len(targets)} (skipping {len(results)} already done)")
for i, img_path in enumerate(targets, 1):
try:
results[img_path.name] = classify_wallpaper(str(img_path))
except Exception as e:
print(f" Failed: {img_path.name} — {e}")
results[img_path.name] = {"category": "Unclassified", "error": str(e)}
# Flush periodically; a single write at the end loses everything on a crash
if i % save_every == 0 or i == len(targets):
with open(output_file, "w", encoding="utf-8") as f:
json.dump(results, f, ensure_ascii=False, indent=2)
print(f" -> saved {i}/{len(targets)}")Two details matter here. First, entries containing error are excluded from the "done" set, so an image that failed on a transient rate limit gets another attempt instead of being frozen as "Unclassified" forever.
Second, the periodic flush. Writing once at the end is tempting, but a crash at image 480 costs you 480 requests. Flushing every 20 caps the loss at 19. That trade is worth far more than the saved file I/O.
If you add the resize step from the previous section, change classify_wallpaper() to accept bytes rather than a path — you can hand it the output of resize_for_classification() directly and read each file only once.
The Cost Math That Actually Matters
Running indie apps since 2014 has taught me that the right metric for automation tools isn't absolute cost — it's opportunity cost.
The Claude API classification runs cost me roughly $2–3 per month across all my apps. The alternative is 4–6 hours of manual tagging per month. By any measure, this is one of the highest-ROI investments in my workflow.
What I didn't expect was the secondary benefit: consistency. When I tagged manually over multiple sessions, the same borderline image might end up in different categories depending on my mood or how tired I was. The API is consistent every time, which means the category distribution in the app is more reliable.
A Practical Starting Point
If you want to try this for image classification of any kind — not just wallpapers — I'd suggest one thing before writing code: define your categories in complete sentences, not just labels.
Instead of: "Nature"
Try: "Nature & Scenery: landscapes, mountains, forests, sky, and other natural environments as the primary subject"
That single change improved Haiku's accuracy by about 4% in my testing. Claude responds well to descriptive context, and category definitions are no exception.
One caveat on the confidence field: it is self-reported, not a calibrated probability. I have seen wrong answers come back at 0.9. I treat it as a sort order for review rather than a guarantee of correctness — on a day when I only have twenty minutes, I review the lowest-scoring images for twenty minutes and stop. That has served me better than trusting a fixed threshold.
Start with 50 images and run them through both Haiku and Sonnet. Compare where they agree and disagree. The disagreements will tell you exactly where your category definitions need refinement — and most of the time, refining the prompt is cheaper than upgrading the model.