⬡ API & SDK/2026-09-06Intermediate

The translation read perfectly and still crashed at runtime

When you translate app strings with the Claude API, the meaning can be right while the format specifiers quietly break. Here is a severity-aware acceptance check and a repair loop that only re-translates the broken lines.

Claude API¹²⁶ localization⁴i18nquality gateindie developer²¹

✦ Premium Article

I was a few days into a 5% staged rollout of a wallpaper app when crash reports started arriving from one language and nowhere else. The only thing I had touched was a batch of in-app strings. No code had changed.

I read the translations top to bottom. Every line was fluent and said the right thing. Nothing looked wrong as prose.

The problem only surfaced when I stopped reading sentences and lined up the symbols instead. The source had %1$@. The translation had %@.

As long as you evaluate a translation as writing, format-specifier damage stays invisible. It isn't a vocabulary problem. It's a syntax problem.

What broke was argument binding, not meaning

A format specifier is a hole where a value gets injected at runtime. Both String(format:) on iOS and String.format on Android assign arguments by counting those holes and checking their types.

A translation model translates sentences. The holes are part of the sentence, so when word order gets rearranged, positional indexes get dropped, or the token gets rewritten to whichever platform convention the model considers idiomatic. There is no malice and no carelessness involved — it falls out of producing a natural sentence.

I started by checking what actually happens when arguments are short or mistyped. Here is real output from Python's % formatting:

"%s / %s" with one argument: TypeError: not enough arguments for format string
"%d" with a string:          TypeError: %d format: a real number is required, not str
"%(n)s" with an empty dict:  KeyError: 'n'
"%s" with two arguments:     succeeded -> 'a'   <- no exception

That last line is the important one. Extra holes crash. Extra arguments do not. Android's String.format has the same asymmetry: referencing an argument that was never supplied raises MissingFormatArgumentException, a type mismatch raises IllegalFormatConversionException, but leaving an argument unused raises nothing at all.

So "a specifier disappeared" and "a specifier appeared" are two different incidents. The first silently loses information. The second takes the device down. Collapsing both into a single "mismatch" verdict means you either fail to stop the dangerous one, or you block releases over the harmless one.

Six ways it breaks

Everything I collected fell into six shapes.

Shape	Source	Translation	Effect
Positional index dropped	`%1$@ items in %2$d s`	`%@ items in %d s`	Order becomes language-dependent
Platform convention swapped	`%1$@`	`%1$s`	Invalid on iOS
Specifier missing	`%1$@ to %2$@`	`Moved %1$@`	Information disappears
Specifier added	`Saved %1$@`	`Saved %1$@ to %2$@`	Crashes at runtime
Glyph corrupted	`%1$@`	`％1$@` / `% 1$@`	No longer parsed as a specifier
Escape lost	`%1$d%%`	`%1$d%`	A bare % survives into the format string

Glyph corruption shows up more often than I expected whenever a Japanese keyboard sits anywhere in the pipeline. A full-width ％ is essentially indistinguishable by eye.

✦

Thank you for reading this far.

Continue Reading

What follows includes implementation code, benchmarks, and practical content we hope you'll find useful. This site runs without ads — server and development costs are supported entirely by members like you. If it's been helpful, we'd be truly grateful for your support.

WHAT YOU'LL LEARN

✦You'll be able to stop format-specifier damage before shipping, even when the translated text reads perfectly

✦You'll be able to treat a dropped specifier and an added one as different incidents, because only one of them crashes

✦You'll be able to add a masked, line-scoped repair loop to a translation pipeline you already run

Secure payment via Stripe · Cancel anytime

✦

Unlock This Article

Get full access to the rest of this article. Buy once, read anytime. This site is ad-free — your support goes directly toward keeping it running.

Unlock all articles with Membership →

Pull out the specifiers first

The first step is extracting specifiers mechanically. One extractor handles iOS %1$@, Android %1$s, and named {count} placeholders.

# fmtcheck.py — extract format specifiers and compare them with severity
import re
from collections import Counter
 
PRINTF = re.compile(
    r"%(?:(\d+)\$)?([-+ 0#]*)(\d+|\*)?(?:\.(\d+|\*))?(?:hh|h|ll|l|q|z|t|L)?([@sdiufFeEgGxXoc%])"
)
BRACE = re.compile(r"\{([^{}\s]+)\}")   # deliberately wide: translators do translate what's inside
BROKEN_GLYPH = re.compile(r"[％＄＠]|%\s+\d+\s*\$|%\s+[@sd]")
LITERAL_PCT = re.compile(r"%%")
 
NUMERIC = set("diufFeEgGxXo")
 
def tokens(text):
    """Return a list of (positional index, conversion char). Index is None when absent."""
    out = []
    for m in PRINTF.finditer(text):
        if m.group(5) == "%":
            continue                      # %% is a literal and consumes no argument
        out.append((int(m.group(1)) if m.group(1) else None, m.group(5)))
    for m in BRACE.finditer(text):
        out.append((m.group(1), "{}"))
    return out
 
def _kind(conv):
    if conv == "{}":
        return "named"
    return "number" if conv in NUMERIC else "text"

Two details in that regex mattered.

First, %% must always be skipped. A literal percent consumes no argument, so counting it as a specifier makes source and translation permanently disagree.

Second, I widened the brace pattern. My first version restricted names to [A-Za-z_][A-Za-z0-9_]*. When a line came back with {count} rendered as {件数}, that version only saw "the source name disappeared" and completely missed "an unknown name appeared." Additions are the crashing side, so that was exactly the wrong thing to miss.

The comparison rule depends on positional indexes

This was the fork in the implementation.

If the source uses positional indexes — the 1$ in %1$@ — the translation is free to reorder them. The index points at the argument, so order carries no meaning. Only count and type matter.

If there are no positional indexes, the logic inverts. %@ and %d consume arguments in the order they appear, so the sequence itself is the argument binding. Reorder it and the types swap, and it crashes.

def check(src, dst):
    findings = []
    s, d = tokens(src), tokens(dst)
 
    # Literal %% never shows up in tokens, so count it separately
    ls, ld = len(LITERAL_PCT.findall(src)), len(LITERAL_PCT.findall(dst))
    if ls != ld:
        findings.append(("crash", "literal_percent", (ls, ld)))
 
    g = BROKEN_GLYPH.search(dst)
    if g:
        findings.append(("crash", "broken_glyph", g.group(0)))
 
    if any(a is not None for a, _ in s):
        # Positional: order is free, count and type are not
        if any(a is None for a, _ in d):
            findings.append(("crash", "lost_argnum", [t for t in d if t[0] is None]))
        cs, cd = Counter(s), Counter(d)
        for tok in (cd - cs).elements():
            # Reusing an existing index is legal, so demote that case to review
            findings.append(("review" if tok in cs else "crash", "extra", tok))
        for tok in (cs - cd).elements():
            findings.append(("loss", "missing", tok))
        types_s = {a: c for a, c in s if a is not None}
        types_d = {a: c for a, c in d if a is not None}
        for a, c in types_d.items():
            if a in types_s and _kind(types_s[a]) != _kind(c):
                findings.append(("crash", "type_mismatch", (a, types_s[a], c)))
    else:
        # Non-positional: the sequence is the binding
        seq_s = [c for _, c in s]
        seq_d = [c for _, c in d]
        if len(seq_d) > len(seq_s):
            findings.append(("crash", "too_many", seq_d))
        elif len(seq_d) < len(seq_s):
            findings.append(("loss", "too_few", seq_d))
        elif seq_s != seq_d:
            findings.append(("crash", "sequence_type_shift", seq_d))
    return findings
 
def worst(findings):
    for level in ("crash", "loss", "review"):
        if any(f[0] == level for f in findings):
            return level
    return "ok"

Three severity levels exist so the pipeline keeps moving. crash blocks the release. loss gets accepted and sent back to a human. review only lands on someone's desk; it never blocks.

Comparing multisets rather than sets was also a measured decision. Here is the actual output:

src='%1$@ to %2$@'
 dst='Moved %2$@ to %1$@ and %1$@'
  set comparison=equal / multiset comparison=different

A set comparison misses the duplicated index entirely. Reusing %1$@ is legal and will not crash, but a repetition that the source never had is not something I want passing in silence. That is what the middle severity is for.

What the fixtures measured

I ran seventeen source/translation pairs through it. This is the raw output:

id                   severity  first finding
ok_positional_swap   ok      —
ok_same              ok      —
ok_percent_literal   ok      —
ok_brace             ok      —
meaning_swap         ok      —
lost_argnum          crash   ('crash', 'lost_argnum', [(None, '@'), (None, 'd')])
platform_token_swap  crash   ('crash', 'extra', (1, 's'))
dropped_one          loss    ('loss', 'missing', (2, '@'))
added_one            crash   ('crash', 'extra', (2, '@'))
type_mismatch        crash   ('crash', 'extra', (1, '@'))
fullwidth            crash   ('crash', 'broken_glyph', '％')
space_injected       crash   ('crash', 'broken_glyph', '% 1$')
seq_reordered        crash   ('crash', 'sequence_type_shift', ['d', '@'])
brace_translated     crash   ('crash', 'extra', ('件数', '{}'))
brace_dropped        loss    ('loss', 'missing', ('count', '{}'))
percent_unescaped    crash   ('crash', 'literal_percent', (1, 0))
reuse_positional     review  ('review', 'extra', (1, '@'))
 
totals: {'ok': 5, 'crash': 9, 'loss': 2, 'review': 1} / 17 cases

The percent_unescaped row taught me the most. A %% collapsing into % is not always visible to specifier comparison. Four translations of the same source:

'Free %1$d%% left'     tokens=[(1, 'd')]              verdict=ok
'Free %1$d% left'      tokens=[(1, 'd'), (None, 'e')] verdict=crash
'Free (%1$d%)'         tokens=[(1, 'd')]              verdict=ok    <- slipped through
'Free %1$d%'           tokens=[(1, 'd')]              verdict=ok    <- slipped through

When a bare % happens to be followed by a conversion character such as e, it gets parsed as a specifier and caught. When the next character is ) or end of line, it sails past. The same defect was detectable or invisible depending on the next character. That is when I added a separate count of %% occurrences, which is where the literal_percent finding above comes from.

The gate belongs at ingestion, not in CI

I planned to put this in CI and changed my mind. By the time CI runs, the translations are already in the repository, and the unit of rejection becomes a commit.

Check them the moment they come back and the unit of rejection is a line.

# gate.py — run this immediately after receiving a translation file
import re, sys
from fmtcheck import check, worst
 
STRINGS = re.compile(r'^\s*"((?:[^"\\]|\\.)*)"\s*=\s*"((?:[^"\\]|\\.)*)"\s*;', re.M)
 
def load_strings(path):
    with open(path, encoding="utf-8") as f:
        return dict(STRINGS.findall(f.read()))
 
def run(base_path, target_path):
    base, target = load_strings(base_path), load_strings(target_path)
    rows = []
    for key, src in base.items():
        dst = target.get(key)
        if dst is None:
            rows.append((key, "loss", [("loss", "missing_key", key)]))
            continue
        f = check(src, dst)
        rows.append((key, worst(f), f))
    return rows
 
if __name__ == "__main__":
    rows = run(sys.argv[1], sys.argv[2])
    order = {"crash": 0, "loss": 1, "review": 2, "ok": 3}
    rows.sort(key=lambda r: order[r[1]])
    blocked = sum(1 for _, level, _ in rows if level == "crash")
    for key, level, f in rows:
        print(f"[{level:<6}] {key}: {f if f else '—'}")
    print(f"\n{blocked} crash / {len(rows)} total")
    sys.exit(1 if blocked else 0)

Real output against a four-line German file:

[crash ] storage: [('crash', 'broken_glyph', '％'), ('loss', 'missing', (1, 'd'))]
[crash ] retry: [('crash', 'sequence_type_shift', ['d', '@'])]
[loss  ] move_done: [('loss', 'missing', (2, '@'))]
[ok    ] add_done: —
 
2 crash / 4 total
exit=1

One note on add_done being ok. Its translation is Added %2$d wallpapers in %1$@ seconds, which reverses the order of the source. Because the indexes are positional, that is a correct translation. An implementation that rejects every reordering would block good work right here.

Masking helps, and the masks themselves break

For re-translation I replace the specifiers with opaque markers before sending them. Left as %1$@, they get folded into the sentence.

# mask.py — park the specifiers before handing text to the model
import re
from fmtcheck import PRINTF, BRACE
 
TOKEN = "§{}§"                       # unlikely to be translated, unlikely to collide
BACK  = re.compile(r"§(\d+)§")
 
def mask(text):
    slots, idx = [], 0
    def take(m):
        nonlocal idx
        if m.group(0) == "%%":
            return m.group(0)
        slots.append(m.group(0)); idx += 1
        return TOKEN.format(idx - 1)
    out = PRINTF.sub(take, text)
    out = BRACE.sub(take, out)
    return out, slots
 
def unmask(text, slots):
    missing = []
    def put(m):
        i = int(m.group(1))
        if i >= len(slots):
            missing.append(i); return m.group(0)
        return slots[i]
    restored = BACK.sub(put, text)
    used = {int(m.group(1)) for m in BACK.finditer(text)}
    return restored, sorted(set(range(len(slots))) - used), missing

I expected masking to make the problem go away. Running five return shapes through it showed otherwise:

masked : §0§ wallpapers added in §1§ seconds (free §2§%%)
parked : ['%1$@', '%2$d', '%3$d']
 
returned verbatim     unused slots = []  / unknown markers = []
reordered             unused slots = []  / unknown markers = []
one marker dropped    unused slots = [1] / unknown markers = []
marker translated     unused slots = [0] / unknown markers = []
marker invented       unused slots = []  / unknown markers = [3]

"Marker translated" is the case where §0§ comes back as something like ⟪0⟫. It cannot be restored, and searching for § will never find it. Detection still works: count the parked slots that nobody used. Whether the marker vanished or got rewritten is indistinguishable, and it turned out I never needed to distinguish them — both mean "do not accept this line."

Since adopting masking I always inspect the return values of unmask. Taking only the restored string and moving on throws the entire check away.

Re-translate the broken lines only

Collect the crash lines, mask them, send them back. Skipping a full re-run isn't only about cost. It's about not creating a fresh opportunity for a line that was already fine to break.

# repair.py — re-translate only the crash lines, at most two rounds
import json, os
from anthropic import Anthropic
from fmtcheck import check, worst
from mask import mask, unmask
 
client = Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])
MODEL = "claude-sonnet-5"
 
SYSTEM = (
    "You translate short UI strings. The input contains opaque markers like §0§. "
    "Copy every marker verbatim, keep the same count, never translate or reformat them. "
    "Return JSON only: an object mapping each key to the translated string."
)
 
def translate_batch(items, target_lang):
    masked, slotmap = {}, {}
    for k, src in items.items():
        masked[k], slotmap[k] = mask(src)
 
    msg = client.messages.create(
        model=MODEL,
        max_tokens=2000,
        system=SYSTEM,
        messages=[{
            "role": "user",
            "content": f"Target language: {target_lang}\n"
                       f"Strings:\n{json.dumps(masked, ensure_ascii=False, indent=2)}",
        }],
    )
    raw = json.loads(msg.content[0].text)
 
    out = {}
    for k, translated in raw.items():
        restored, unused, unknown = unmask(translated, slotmap[k])
        # Drop the line before anyone reads the prose
        out[k] = None if (unused or unknown) else restored
    return out
 
def repair(base, target, lang, max_rounds=2):
    fixed, log = dict(target), []
    for rnd in range(1, max_rounds + 1):
        broken = {k: base[k] for k in base
                  if worst(check(base[k], fixed.get(k, ""))) == "crash"}
        if not broken:
            log.append((rnd, 0, "stopped: no crash left"))
            break
        got = translate_batch(broken, lang)
        applied = 0
        for k, v in got.items():
            if v is None:
                continue
            if worst(check(base[k], v)) != "crash":   # never accept an unfixed line
                fixed[k] = v
                applied += 1
        log.append((rnd, len(broken), f"accepted {applied}"))
    still = [k for k in base if worst(check(base[k], fixed.get(k, ""))) == "crash"]
    return fixed, still, log

Output from a local run with translate_batch swapped for a stub, so only the control flow is exercised — two lines fixed in the first round, one in the second:

round 1: 3 candidates -> accepted 2
round 2: 1 candidates -> accepted 1
remaining crash: []
  add_done  [ok    ] Added %1$@ wallpapers in %2$d seconds
  storage   [ok    ] Uses %1$d%% of free space
  retry     [ok    ] Tried %@ times, %d attempts
  move      [loss  ] Moved %1$@

move staying at loss is the design working. Rewriting a line that merely lost information means letting a machine decide what the sentence was supposed to say. Non-crashing damage goes back to a person.

The round cap comes from the same place. A line that survives two rounds has a problem in the instructions or in the source string itself. A third round just circles the same ground.

What this check cannot catch

I'd rather be plain about the limits. This only stops the damage that crashes.

src='Copied from %1$@ to %2$@'
dst='Copied from %2$@ to %1$@'
verdict=ok

Counts match, types match, nothing crashes. The arguments are simply swapped, so the screen tells the user it copied from the destination to the source. A machine passes it; a person spots it instantly.

That is why review exists as a level. Rather than deciding everything mechanically, let the machine assert only what it can assert with certainty. The same boundary shows up in my notes on layered validation for structured responses. For the surrounding pipeline there's running multilingual translation in production, and for the image side, localizing a screenshot PSD into fifteen languages.

Running localized apps as an indie developer, it's easy to spend all your attention on translation quality. I spent far too long tuning glossaries and style instructions, and the results were unremarkable. A glossary aligns vocabulary. It does nothing for syntax.

People judge whether a translation reads well; machines judge whether the specifiers are identical. Deciding that one line first is what finally let me ship in the evening without watching the crash dashboard.

One next step

Take one translation output you already have and write ten lines that print the specifiers from the source next to the specifiers from the translation. The tokens() function alone is enough. Severity levels and repair loops can wait until you've seen those two lists side by side.

That is where I started, too. Thank you for reading this far.

Thank You for Reading

Claude Lab is ad-free, supported entirely by members like you. We publish practical guides daily with implementation code, benchmarks, and production-ready patterns. If you've found it useful, we'd love to have you on board.