The Line That Disappears at 100K: Measuring What Tool-Output Spill Actually Keeps

The lines that survive, and the one that never does

Next I counted how many lines fit inside the first 100,000 characters.

Survival ranges from 8.3% to 66.8%. That much I expected.

The problem is that these numbers barely matter in practice.

Line 1,654 — the last line of the find output — reads:

658964 total

That line, plus the largest few files by size, was the entire reason I ran the command. And by definition, the last line is the first casualty of a head cut.

Here is the final line that actually made it into the preview:

   9377 content/articles/en/claude-code/claude-code-gate-and...

An unremarkable file sitting alphabetically around en/claude-code/. That is where the preview stops.

This is where my expectation was exactly backwards. I had understood "large output means the model only sees part of it." The reality is closer to: large output means the model only sees the least useful part.

Command-line tools put their verdict at the end. total from wc. N files changed from git. N passing, M failing from a test runner. Whether detail survives at 8% or 67%, the summary line survived at 0% in both cases. A head cut structurally targets the single densest line in the output.

The agent naming the wrong file was not a hallucination. It gave the most plausible answer available in the material it was handed.

A character cut destroys structure

I tried one payload that is not line-oriented: articles.json, the article metadata index, at 822,582 bytes.

Take the first 100,000 characters and try to parse them.

import json, pathlib
 
raw = pathlib.Path("src/data/articles.json").read_text()
print(f"bytes={len(raw)}  ({len(raw)/100_000:.2f}x)")   # bytes=822582  (8.23x)
 
cut = raw[:100_000]
try:
    json.loads(cut)
    print("head-100K only: parsed")
except json.JSONDecodeError as e:
    print(f"head-100K only: failed -> {e}")
    print(f"last 40 chars before the cut: ...{cut[-40:]!r}")

Result:

bytes=822582  (8.23x)
head-100K only: failed -> Expecting property name enclosed in double quotes: line 3087 column 6
last 40 chars before the cut: ...'ull,\n      "author": "Claude Lab",\n     '

The cut lands on a character index — not a line, not a record. It sliced an object open between two keys.

Line-oriented text degrades gracefully. Those 466 surviving grep lines each keep their full meaning. Structured data has no such property. It is not that 12% of the payload survived; it is that a broken string survived.

If you hand an agent a tool that emits raw JSON, the safe assumption is that crossing the boundary is a total loss, not a partial one.

The boundary moves on its own

Look at the last row again: ls -R content at 87,135 bytes, or 0.87x. No spill today.

Each article adds about 110.2 bytes to that particular output.

# current value, and how many more articles until the boundary
now=$(ls -R content | wc -c)
ja=$(find content/articles/ja -name '*.mdx' | wc -l)
python3 -c "
n, a = $now, $ja
per = n / a
need = (100_000 - n) / per
print(f'now {n} bytes / {a} articles / {per:.1f} bytes per article')
print(f'{need:.0f} more articles to cross (~{need/2:.0f} days at 2 per day)')"

now 87135 bytes / 791 articles / 110.2 bytes per article
117 more articles to cross (~58 days at 2 per day)

In roughly two months this command quietly moves to the other side. Not one line of prompt, tool definition, or code will have changed, yet the nature of what the agent receives will have.

This is the failure mode I find hardest to defend against. Regression tests pass. The tool call returns success. The only thing that changes is whether the answer is right. And the change is not gradual — it arrives all at once, on the day the threshold is crossed.

The output envelope: push decisions to the front

The fix turned out to be simple once I stopped fighting the truncation. Assume the head cut, and rebuild the output around it.

Put what a decision needs at the front, let detail flow behind it, and always write the full text to a file yourself so the path travels with the preview. An envelope between the tool and the model.

#!/usr/bin/env python3
"""outbox.py — repack tool output so decision material sits at the front.
 
  usage:  <command> | python3 outbox.py --label build --tail 20
  Reads stdin, emits header + signal lines + tail + head excerpt on stdout.
  The full text always goes to a file, whose path is carried in the header.
"""
import sys, os, re, json, hashlib, pathlib, argparse
 
SPILL_LIMIT = 100_000     # threshold at which full output spills to a file
HEADER_BUDGET = 2_000     # characters reserved for the decision header
EXCERPT_BUDGET = 12_000   # characters reserved for the head excerpt
 
# Default signal set. Override with --signal to match your test runner's vocabulary.
ERROR_RE = re.compile(r"\b(error|failed|failure|fatal|panic|exception)\b", re.I)
 
 
def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--label", default="tool-output")
    ap.add_argument("--dir", default=os.environ.get("OUTBOX_DIR", "/tmp/outbox"))
    ap.add_argument("--signal", default=None, help="regex for important lines")
    ap.add_argument("--tail", type=int, default=20, help="lines always kept from the end")
    args = ap.parse_args()
 
    raw = sys.stdin.read()
    lines = raw.splitlines()
 
    # Persist first. Do this later and a mid-run crash leaves you with nothing.
    digest = hashlib.sha256(raw.encode("utf-8", "replace")).hexdigest()[:12]
    outdir = pathlib.Path(args.dir)
    outdir.mkdir(parents=True, exist_ok=True)
    full = outdir / f"{args.label}-{digest}.txt"
    full.write_text(raw, encoding="utf-8", errors="replace")
 
    pat = re.compile(args.signal, re.I) if args.signal else ERROR_RE
    hits = [(i, l) for i, l in enumerate(lines, 1) if pat.search(l)]
 
    header = {
        "label": args.label,
        "bytes": len(raw),
        "lines": len(lines),
        "spill_ratio": round(len(raw) / SPILL_LIMIT, 2),
        "signal_hits": len(hits),
        "full_output": str(full),          # point the model here for the rest
        "tail_lines_included": min(args.tail, len(lines)),
    }
 
    parts = ["<<<OUTBOX-HEADER",
             json.dumps(header, ensure_ascii=False, indent=2),
             "OUTBOX-HEADER"]
 
    if hits:
        parts.append(f"--- signal lines (first 40 of {len(hits)}) ---")
        parts += [f"{i}: {l[:300]}" for i, l in hits[:40]]
 
    # Where CLI tools put their verdict. Drop this and the envelope is pointless.
    parts.append(f"--- tail {header['tail_lines_included']} lines ---")
    parts += lines[-args.tail:]
 
    body = "\n".join(parts)
    if len(body) > HEADER_BUDGET + EXCERPT_BUDGET:
        body = body[: HEADER_BUDGET + EXCERPT_BUDGET] + "\n--- excerpt truncated; read full_output ---"
 
    sys.stdout.write("\n".join([
        body,
        f"--- head excerpt ({EXCERPT_BUDGET} chars max) ---",
        raw[:EXCERPT_BUDGET],
    ]))
 
 
if __name__ == "__main__":
    main()

The same two commands, measured through the envelope:

The find case folds to 11.5% and keeps 658964 total in the tail block. The agent can now answer with the total in hand.

The grep case mattered more operationally. An output 12x over the boundary came down to 31,279 bytes, so no spill happens at all. The read-the-file round trip disappears, while the full text stays one path away whenever it is genuinely needed.

Structured data needs a different envelope

The text envelope does not work on JSON. Appending 20 trailing lines does not produce a parseable document.

For structured data the summary has to describe shape rather than values: item count, key inventory, and a small set of samples at full fidelity. And the envelope should verify its own output parses before it returns.

#!/usr/bin/env python3
"""json_outbox.py — fold a large JSON payload while staying valid JSON."""
import json, sys, hashlib, pathlib, collections
 
 
def envelope(path, keep=8, outdir="/tmp/outbox"):
    raw = pathlib.Path(path).read_text()
    data = json.loads(raw)
 
    # Top level may be an object; take the first array we find inside it.
    items = data if isinstance(data, list) else next(
        (v for v in data.values() if isinstance(v, list)), [])
 
    digest = hashlib.sha256(raw.encode()).hexdigest()[:12]
    outdir = pathlib.Path(outdir)
    outdir.mkdir(parents=True, exist_ok=True)
    full = outdir / f"json-{digest}.json"
    full.write_text(raw)
 
    # Scanning every item is needlessly slow on large files; 2000 was enough
    # to capture the full key inventory in every payload I tested.
    keys = collections.Counter()
    for it in items[:2000]:
        if isinstance(it, dict):
            keys.update(it.keys())
 
    body = {
        "header": {
            "source": str(path),
            "bytes": len(raw),
            "item_count": len(items),
            "keys": [k for k, _ in keys.most_common()],
            "full_output": str(full),
            "sample_policy": f"first {keep} items, full fidelity",
        },
        "sample": items[:keep],
    }
    return json.dumps(body, ensure_ascii=False, indent=1), len(raw)
 
 
if __name__ == "__main__":
    out, n = envelope(sys.argv[1])
    json.loads(out)   # self-check: the envelope must always be parseable
    print(f"raw={n} envelope={len(out)} ratio={len(out)/n*100:.2f}% parse=OK")

raw=822582 envelope=5688 ratio=0.69% parse=OK

822,582 bytes down to 5,688 — 0.69% — and the result is valid JSON, in contrast to the raw head cut returning JSONDecodeError.

Here I hit a second expectation that turned out to be wrong.

I started with roughly 50 sample items, on the assumption that fewer samples meant less information. In use, 8 and 50 produced no difference in answer quality. What the agent draws from this envelope is which keys exist and how many records there are — not the contents of any individual record.

The sample exists to show shape, not to convey content. Cutting it to 8 freed budget that was better spent on making the key inventory complete.

Deciding where to apply it

Not every tool needs an envelope. My rule of thumb:

1. Wrap it when output size scales with the size of the thing being inspected: grep, find, git log, log retrieval, list endpoints. Something at 0.5x today will pass 2x within a year.
2. Skip it when output is constant-size: reading a config value, fetching one record, checking status. The header ends up larger than the payload and makes things harder to read.
3. Raise --tail when order carries meaning. For time-ordered logs the end is the present. The default 20 was too thin for deployment logs, where I moved to 60.

One more note on spill_ratio in the header: it exists to make distance-to-boundary visible during normal operation. Any value above 0.9 in the logs means that tool is about to cross. Learning that two months early is far easier than discovering it after the fact.

Running automation as an indie developer, every one of these quiet-failure paths I close visibly widens what I can hand off. Bugs that fail loudly have cost me far less time than bugs that return success while being wrong.

Where to measure next

If you want to try this, start by running your own tool set as-is and counting bytes.

for c in "grep -rn TODO ." "find . -type f -exec wc -c {} +" "git log --stat -50"; do
  n=$(eval "$c" 2>/dev/null | wc -c)
  printf "%-40s %10d  %5.2fx\n" "${c:0:38}" "$n" "$(python3 -c "print($n/100000)")"
done

Anything above 1.0 has already lost its summary line. Anything below it is a question of when, and that is worth estimating today.

References

• Claude Platform release notes
• Claude Code changelog