The Line That Disappears at 100K: Measuring What Tool-Output Spill Actually Keeps

The lines that survive, and the one that never does

Next I counted how many lines fit inside the first 100,000 characters.

Survival ranges from 8.3% to 66.8%. That much I expected.

The problem is that these numbers barely matter in practice.

Line 1,654 — the last line of the find output — reads:

658964 total

That line, plus the largest few files by size, was the entire reason I ran the command. And by definition, the last line is the first casualty of a head cut.

Here is the final line that actually made it into the preview:

   9377 content/articles/en/claude-code/claude-code-gate-and...

An unremarkable file sitting alphabetically around en/claude-code/. That is where the preview stops.

This is where my expectation was exactly backwards. I had understood "large output means the model only sees part of it." The reality is closer to: large output means the model only sees the least useful part.

Command-line tools put their verdict at the end. total from wc. N files changed from git. N passing, M failing from a test runner. Whether detail survives at 8% or 67%, the summary line survived at 0% in both cases. A head cut structurally targets the single densest line in the output.

The agent naming the wrong file was not a hallucination. It gave the most plausible answer available in the material it was handed.

I had the units wrong: the boundary counts characters, wc -c counts bytes

Rebuilding the two tables above after publishing, I noticed the counts were not measured the same way.

The grep, find, and ls -R rows came from wc -c, which returns bytes. The JSON row came from Python's len(raw), which returns characters. And the spill boundary is drawn at 100,000 characters, not bytes.

For output containing Japanese text, those two numbers do not agree. I counted both against the same repository (it has grown from 791 to 803 Japanese articles since publication, so absolute values shift slightly).

find and ls -R emit nothing but file paths, so bytes and characters match exactly. The gap appears in grep, which carries article prose, and in the JSON, which carries Japanese titles and descriptions. A Japanese character is three bytes in UTF-8, so the share of Japanese text lands directly in the ratio.

The consequences came in two layers.

First, the ratios. I described the grep output as 12.01x over the boundary. Measured in the same unit the boundary uses, it is 9.45x. I had overstated it by about 29%.

Second, and more consequential, the survival rates. Cutting at the first 100,000 bytes leaves 466 lines. Cutting at the first 100,000 characters leaves 607. The second one is what actually reaches the model.

The 8.3% figure I published for grep should have been 10.7%. The find row is pure ASCII, so its 66.8% (65.7% on re-measurement) is unaffected.

The error direction is at least the safe one. Counting bytes makes output look closer to the boundary than it is, so nothing that has already crossed goes unnoticed. It flips the other way only when you reason from the safe side — "still only 0.9x, we're fine". The more multibyte text an output carries, the further short of the real position a byte count lands.

The fix is one flag.

# Wrong: bytes. Runs high on any output carrying multibyte text
grep -rn 'claude' content/articles/ja --include='*.mdx' | wc -c   # 1218722
# Right: characters, the same unit the spill boundary uses
grep -rn 'claude' content/articles/ja --include='*.mdx' | wc -m   #  945213

One caveat: wc -m is locale-dependent and returns the byte count under LC_ALL=C. These measurements ran under LANG=C.UTF-8. In CI or a bare container, check LANG before trusting wc -m, or count with Python's len() instead.

The envelope header further down inherits the same mistake. Its "bytes": len(raw) key actually holds a character count. The revised version below reports chars and bytes separately and compares chars against the boundary.

A character cut destroys structure

I tried one payload that is not line-oriented: articles.json, the article metadata index — 822,582 characters, or 978,702 bytes, in the units established above.

Take the first 100,000 characters and try to parse them.

import json, pathlib
 
raw = pathlib.Path("src/data/articles.json").read_text()
print(f"bytes={len(raw)}  ({len(raw)/100_000:.2f}x)")   # bytes=822582  (8.23x)
 
cut = raw[:100_000]
try:
    json.loads(cut)
    print("head-100K only: parsed")
except json.JSONDecodeError as e:
    print(f"head-100K only: failed -> {e}")
    print(f"last 40 chars before the cut: ...{cut[-40:]!r}")

Result:

bytes=822582  (8.23x)
head-100K only: failed -> Expecting property name enclosed in double quotes: line 3087 column 6
last 40 chars before the cut: ...'ull,\n      "author": "Claude Lab",\n     '

The cut lands on a character index — not a line, not a record. It sliced an object open between two keys.

Line-oriented text degrades gracefully. Those 466 surviving grep lines each keep their full meaning. Structured data has no such property. It is not that 12% of the payload survived; it is that a broken string survived.

If you hand an agent a tool that emits raw JSON, the safe assumption is that crossing the boundary is a total loss, not a partial one.

The boundary moves on its own

Look at the last row again: ls -R content at 87,135 bytes, or 0.87x. No spill today.

Each article adds about 110.2 bytes to that particular output.

# current value, and how many more articles until the boundary
now=$(ls -R content | wc -c)
ja=$(find content/articles/ja -name '*.mdx' | wc -l)
python3 -c "
n, a = $now, $ja
per = n / a
need = (100_000 - n) / per
print(f'now {n} bytes / {a} articles / {per:.1f} bytes per article')
print(f'{need:.0f} more articles to cross (~{need/2:.0f} days at 2 per day)')"

now 87135 bytes / 791 articles / 110.2 bytes per article
117 more articles to cross (~58 days at 2 per day)

In roughly two months this command quietly moves to the other side. Not one line of prompt, tool definition, or code will have changed, yet the nature of what the agent receives will have.

This is the failure mode I find hardest to defend against. Regression tests pass. The tool call returns success. The only thing that changes is whether the answer is right. And the change is not gradual — it arrives all at once, on the day the threshold is crossed.

Eleven days on from publication, I checked the estimate against reality.

Twelve articles added 1,476 characters. Projecting from the 110.2 characters-per-article average gives 88,457, off by 154 characters — 0.17%. Eleven days is a short window, but it confirms a linear estimate is good enough for this purpose.

The marginal rate tells a slightly different story: 123.0 characters per article, 12% above the average. The average is diluted by the early period when the directory lines existed and the articles did not. For distance-to-boundary, dividing by the recent marginal rate is the more conservative choice.

That puts the crossing in late September on the marginal rate, end of September on the average. The original "about 58 days" pointed at late September too, so eleven days of real data barely moved the landing point. Having an estimate at all mattered more than its precision.

The output envelope: push decisions to the front

The fix turned out to be simple once I stopped fighting the truncation. Assume the head cut, and rebuild the output around it.

Put what a decision needs at the front, let detail flow behind it, and always write the full text to a file yourself so the path travels with the preview. An envelope between the tool and the model.

#!/usr/bin/env python3
"""outbox.py — repack tool output so decision material sits at the front.
 
  usage:  <command> | python3 outbox.py --label build --tail 20
  Reads stdin, emits header + signal lines + tail + head excerpt on stdout.
  The full text always goes to a file, whose path is carried in the header.
"""
import sys, os, re, json, hashlib, pathlib, argparse
 
SPILL_LIMIT = 100_000     # threshold at which full output spills to a file
HEADER_BUDGET = 2_000     # characters reserved for the decision header
EXCERPT_BUDGET = 12_000   # characters reserved for the head excerpt
 
# Default signal set. Override with --signal to match your test runner's vocabulary.
ERROR_RE = re.compile(r"\b(error|failed|failure|fatal|panic|exception)\b", re.I)
 
 
def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--label", default="tool-output")
    ap.add_argument("--dir", default=os.environ.get("OUTBOX_DIR", "/tmp/outbox"))
    ap.add_argument("--signal", default=None, help="regex for important lines")
    ap.add_argument("--tail", type=int, default=20, help="lines always kept from the end")
    args = ap.parse_args()
 
    raw = sys.stdin.read()
    lines = raw.splitlines()
 
    # Persist first. Do this later and a mid-run crash leaves you with nothing.
    digest = hashlib.sha256(raw.encode("utf-8", "replace")).hexdigest()[:12]
    outdir = pathlib.Path(args.dir)
    outdir.mkdir(parents=True, exist_ok=True)
    full = outdir / f"{args.label}-{digest}.txt"
    full.write_text(raw, encoding="utf-8", errors="replace")
 
    pat = re.compile(args.signal, re.I) if args.signal else ERROR_RE
    hits = [(i, l) for i, l in enumerate(lines, 1) if pat.search(l)]
 
    header = {
        "label": args.label,
        "bytes": len(raw),
        "lines": len(lines),
        "spill_ratio": round(len(raw) / SPILL_LIMIT, 2),
        "signal_hits": len(hits),
        "full_output": str(full),          # point the model here for the rest
        "tail_lines_included": min(args.tail, len(lines)),
    }
 
    parts = ["<<<OUTBOX-HEADER",
             json.dumps(header, ensure_ascii=False, indent=2),
             "OUTBOX-HEADER"]
 
    if hits:
        parts.append(f"--- signal lines (first 40 of {len(hits)}) ---")
        parts += [f"{i}: {l[:300]}" for i, l in hits[:40]]
 
    # Where CLI tools put their verdict. Drop this and the envelope is pointless.
    parts.append(f"--- tail {header['tail_lines_included']} lines ---")
    parts += lines[-args.tail:]
 
    body = "\n".join(parts)
    if len(body) > HEADER_BUDGET + EXCERPT_BUDGET:
        body = body[: HEADER_BUDGET + EXCERPT_BUDGET] + "\n--- excerpt truncated; read full_output ---"
 
    sys.stdout.write("\n".join([
        body,
        f"--- head excerpt ({EXCERPT_BUDGET} chars max) ---",
        raw[:EXCERPT_BUDGET],
    ]))
 
 
if __name__ == "__main__":
    main()

The same two commands, measured through the envelope:

The find case folds to 11.5% and keeps 658964 total in the tail block. The agent can now answer with the total in hand.

The grep case mattered more operationally. An output 12x over the boundary came down to 31,279 bytes, so no spill happens at all. The read-the-file round trip disappears, while the full text stays one path away whenever it is genuinely needed.

When the envelope drops the conclusion: budget from the front, not the back

After running this envelope for a while, I found a defect in it. It discards the line I most wanted to protect, in exactly the situation where I most wanted it protected.

Follow the assembly order. Header, then signal lines (up to 40, each capped at 300 characters), then the tail block, all pushed onto parts — and the whole thing truncated at 14,000 characters at the end. Truncation cuts from the back. So the tail block, appended last and holding the only copy of the conclusion, is the first thing to go.

This bites when signal lines are numerous and long. An error-heavy production build or deployment log has precisely that shape.

I generated a 1,200-line synthetic build log ending in BUILD FAILED: 3 errors, 12 warnings in 48.2s, with 40 error lines, and varied only the length of the signal lines.

The threshold sits between 220 and 240. Past it, the tail block erodes quietly from the back. The more errors there are, the less visible the conclusion becomes.

Notice also that the output plateaus at 26,084 characters. The 14,000 I wrote as a budget only governs body. A 12,000-character head excerpt is appended unconditionally afterwards, so the real output runs to roughly 26,000.

So the envelope was trimming the conclusion at the tail while spending 12,000 characters on the head. An article arguing that head truncation structurally targets the densest line in the output had rebuilt that same structure inside its own fix.

The correction is to reserve budget up front rather than trim from the back.

#!/usr/bin/env python3
"""outbox2.py — reserve the tail block first, then distribute what is left."""
import sys, os, re, json, hashlib, pathlib, argparse
 
SPILL_LIMIT  = 100_000
TOTAL_BUDGET = 14_000     # ceiling for the whole output, head excerpt included
TAIL_BUDGET  = 3_000      # set aside for the tail block before anything else
 
 
def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--label", default="tool-output")
    ap.add_argument("--dir", default=os.environ.get("OUTBOX_DIR", "/tmp/outbox"))
    ap.add_argument("--signal", default=None)
    ap.add_argument("--tail", type=int, default=20)
    args = ap.parse_args()
 
    raw = sys.stdin.read()
    lines = raw.splitlines()
 
    digest = hashlib.sha256(raw.encode("utf-8", "replace")).hexdigest()[:12]
    outdir = pathlib.Path(args.dir)
    outdir.mkdir(parents=True, exist_ok=True)
    full = outdir / f"{args.label}-{digest}.txt"
    full.write_text(raw, encoding="utf-8", errors="replace")
 
    pat = re.compile(args.signal, re.I) if args.signal else re.compile(
        r"\b(error|failed|failure|fatal|panic|exception)\b", re.I)
    hits = [(i, l) for i, l in enumerate(lines, 1) if pat.search(l)]
 
    # 1. Reserve the tail block first. If this gets cut, the envelope has no point
    tail_lines, used = [], 0
    for l in reversed(lines[-args.tail:]):
        if used + len(l) + 1 > TAIL_BUDGET:
            break
        tail_lines.insert(0, l)
        used += len(l) + 1
    tail_block = [f"--- tail {len(tail_lines)}/{min(args.tail, len(lines))} lines ---"] + tail_lines
 
    header = {
        "label": args.label,
        "chars": len(raw),                       # same unit as the boundary; compare on this
        "bytes": len(raw.encode("utf-8")),
        "lines": len(lines),
        "spill_ratio_chars": round(len(raw) / SPILL_LIMIT, 2),
        "signal_hits": len(hits),
        "full_output": str(full),
        "tail_lines_included": len(tail_lines),
        "tail_lines_dropped": min(args.tail, len(lines)) - len(tail_lines),
    }
    head_block = ["<<<OUTBOX-HEADER",
                  json.dumps(header, ensure_ascii=False, indent=2),
                  "OUTBOX-HEADER"]
 
    # 2. Spend the remainder on signal lines; report the overflow as a count
    fixed = len("\n".join(head_block)) + len("\n".join(tail_block)) + 2
    sig_budget = max(0, TOTAL_BUDGET - fixed)
    sig, used, shown = [], 0, 0
    for i, l in hits:
        e = f"{i}: {l[:300]}"
        if used + len(e) + 1 > sig_budget:
            break
        sig.append(e)
        used += len(e) + 1
        shown += 1
    sig_block = ([f"--- signal lines ({shown} of {len(hits)} shown) ---"] + sig) if hits else []
    if hits and shown < len(hits):
        sig_block.append(f"--- {len(hits) - shown} more signal lines omitted; read full_output ---")
 
    # 3. Head excerpt last, and only if there is room left
    body = "\n".join(head_block + sig_block + tail_block)
    room = max(0, TOTAL_BUDGET - len(body))
    out = [body]
    if room > 500:
        out += [f"--- head excerpt ({room} chars) ---", raw[:room]]
    sys.stdout.write("\n".join(out))
 
 
if __name__ == "__main__":
    main()

The same inputs through both versions:

The conclusion line survives in every case and the output settles around 14,000 characters. When signal lines do not fit, the envelope says "35 of 41 shown" and leaves the rest behind full_output. Not letting the model believe it has seen everything turned out to matter about as much as preserving the tail.

Against real output:

Neither of these two loses the conclusion under the old envelope. That is likely why the defect escaped me at publication: it never showed up in the commands I had been measuring. When you test an envelope, do not feed it your ordinary output — feed it the log from the day something broke, dense with signal lines and long ones at that. A safety net verified only on the happy path tends to be missing on the unhappy one.

Structured data needs a different envelope

The text envelope does not work on JSON. Appending 20 trailing lines does not produce a parseable document.

For structured data the summary has to describe shape rather than values: item count, key inventory, and a small set of samples at full fidelity. And the envelope should verify its own output parses before it returns.

#!/usr/bin/env python3
"""json_outbox.py — fold a large JSON payload while staying valid JSON."""
import json, sys, hashlib, pathlib, collections
 
 
def envelope(path, keep=8, outdir="/tmp/outbox"):
    raw = pathlib.Path(path).read_text()
    data = json.loads(raw)
 
    # Top level may be an object; take the first array we find inside it.
    items = data if isinstance(data, list) else next(
        (v for v in data.values() if isinstance(v, list)), [])
 
    digest = hashlib.sha256(raw.encode()).hexdigest()[:12]
    outdir = pathlib.Path(outdir)
    outdir.mkdir(parents=True, exist_ok=True)
    full = outdir / f"json-{digest}.json"
    full.write_text(raw)
 
    # Scanning every item is needlessly slow on large files; 2000 was enough
    # to capture the full key inventory in every payload I tested.
    keys = collections.Counter()
    for it in items[:2000]:
        if isinstance(it, dict):
            keys.update(it.keys())
 
    body = {
        "header": {
            "source": str(path),
            "bytes": len(raw),
            "item_count": len(items),
            "keys": [k for k, _ in keys.most_common()],
            "full_output": str(full),
            "sample_policy": f"first {keep} items, full fidelity",
        },
        "sample": items[:keep],
    }
    return json.dumps(body, ensure_ascii=False, indent=1), len(raw)
 
 
if __name__ == "__main__":
    out, n = envelope(sys.argv[1])
    json.loads(out)   # self-check: the envelope must always be parseable
    print(f"raw={n} envelope={len(out)} ratio={len(out)/n*100:.2f}% parse=OK")

raw=822582 envelope=5688 ratio=0.69% parse=OK

822,582 characters down to 5,688 — 0.69% — and the result is valid JSON, in contrast to the raw head cut returning JSONDecodeError.

Here I hit a second expectation that turned out to be wrong.

I started with roughly 50 sample items, on the assumption that fewer samples meant less information. In use, 8 and 50 produced no difference in answer quality. What the agent draws from this envelope is which keys exist and how many records there are — not the contents of any individual record.

The sample exists to show shape, not to convey content. Cutting it to 8 freed budget that was better spent on making the key inventory complete.

Deciding where to apply it

Not every tool needs an envelope. My rule of thumb:

1. Wrap it when output size scales with the size of the thing being inspected: grep, find, git log, log retrieval, list endpoints. Something at 0.5x today will pass 2x within a year.
2. Skip it when output is constant-size: reading a config value, fetching one record, checking status. The header ends up larger than the payload and makes things harder to read.
3. Raise --tail when order carries meaning. For time-ordered logs the end is the present. The default 20 was too thin for deployment logs, where I moved to 60.

One more note on spill_ratio in the header: it exists to make distance-to-boundary visible during normal operation. Any value above 0.9 in the logs means that tool is about to cross. Learning that two months early is far easier than discovering it after the fact.

Running automation as an indie developer, every one of these quiet-failure paths I close visibly widens what I can hand off. Bugs that fail loudly have cost me far less time than bugs that return success while being wrong.

If you want to try this, start by running your own tool set as-is and counting characters.

for c in "grep -rn TODO ." "find . -type f -exec wc -c {} +" "git log --stat -50"; do
  n=$(eval "$c" 2>/dev/null | wc -m)   # -m, not -c: the boundary is drawn in characters
  printf "%-40s %10d  %5.2fx\n" "${c:0:38}" "$n" "$(python3 -c "print($n/100000)")"
done

Anything above 1.0 has already lost its summary line. Anything below it is a question of when, and that is worth estimating today.

References

• Claude Platform release notes
• Claude Code changelog