Articles/Claude Code

⟐ Claude Code/2026-08-14Advanced

One generated file outweighed all 70 hand-written source files, so I redrew what Claude Code may read

I profiled a repository I actually run to see which areas consume the most context. Here is what I found, the deny rules I settled on, how search selectivity changes the math, and the options I considered but rejected.

Claude Code²⁸⁴ settings.json¹⁰ context¹² cost design⁴repository operations

✦ Premium Article

"Which file decides where the preview gets cut off?" I asked Claude Code that about a site I run.

The answer came back correct — two files, precisely identified. What surprised me was the volume read along the way. It went well past what I had pictured when I said "just go find it."

As an indie developer working alone, there is nobody to point out that kind of friction. So before changing how I explore, I decided to measure the repository itself. You cannot decide what to exclude until you know what weighs what.

The finding that stopped me: a single generated file — one that gets rebuilt on every build — outweighed all seventy hand-written source files combined.

Measure the repository before the session starts

The subject is the claudelab.net repository: a Next.js site holding 806 Japanese and 806 English articles as MDX, with 1,796 files under version control.

I started with a small script that ranks areas by "how heavy is this to read." There is no need to replicate a tokenizer exactly. As long as areas can be compared against each other, the decision is the same. Japanese characters count as roughly one token each; everything else as roughly 3.6 characters per token.

#!/usr/bin/env python3
"""ctxweight.py - rank version-controlled files by how heavy they are to read.
 
Usage:
    python3 ctxweight.py            # whole repository
    python3 ctxweight.py src        # one directory
"""
import collections
import subprocess
import sys
 
PRICE_PER_MTOK = 2.0  # input price in USD per 1M tokens; substitute your own model's
 
 
def estimate_tokens(text: str) -> int:
    """Approximate CJK at ~1 char per token and everything else at ~3.6 chars.
 
    Exact counts depend on the model's tokenizer, so treat this as a relative
    measure -- "area A is N times area B" -- rather than an absolute figure.
    """
    cjk = sum(1 for ch in text if '　' <= ch <= '鿿' or '＀' <= ch <= '￯')
    return int(cjk + (len(text) - cjk) / 3.6)
 
 
def tracked_files(roots):
    # git ls-files means .gitignore'd paths drop out without a hand-kept list
    out = subprocess.check_output(['git', 'ls-files', '-z'] + list(roots)).decode()
    return [f for f in out.split('\0') if f]
 
 
def group(path: str) -> str:
    parts = path.split('/')
    return '/'.join(parts[:2]) if len(parts) > 1 else '(root)'
 
 
def main():
    agg = collections.defaultdict(lambda: [0, 0])  # [tokens, files]
    per_file = []
 
    for path in tracked_files(sys.argv[1:]):
        try:
            text = open(path, encoding='utf-8').read()
        except (UnicodeDecodeError, OSError):
            continue  # binaries such as images are out of scope; skip without noise
        tokens = estimate_tokens(text)
        agg[group(path)][0] += tokens
        agg[group(path)][1] += 1
        per_file.append((tokens, path))
 
    total = sum(v[0] for v in agg.values()) or 1
    print(f"{'area':<28}{'est. tokens':>12}{'files':>7}{'share':>8}{'$/read':>9}")
    for key, (tokens, files) in sorted(agg.items(), key=lambda x: -x[1][0])[:15]:
        print(f"{key:<28}{tokens:>12,}{files:>7,}"
              f"{tokens / total * 100:>7.1f}%{tokens / 1e6 * PRICE_PER_MTOK:>9.3f}")
    print(f"{'total':<28}{total:>12,}"
          f"{sum(v[1] for v in agg.values()):>7,}{100.0:>7.1f}%"
          f"{total / 1e6 * PRICE_PER_MTOK:>9.3f}")
 
    print("\n-- heaviest individual files --")
    for tokens, path in sorted(per_file, reverse=True)[:5]:
        print(f"{tokens:>12,}  {path}")
 
 
if __name__ == '__main__':
    main()

Using git ls-files matters more than it looks. It drops node_modules and .next for free, and a hand-maintained exclusion list always drifts out of date.

Run against the whole repository:

Area	Est. tokens	Files	Share
content/articles	7,666,927	1,612	91.0%
src/generated	289,603	2	3.4%
repository root	162,418	15	1.9%
content/blog	157,421	70	1.9%
src/app	80,278	36	1.0%
src/components	37,332	22	0.4%
Total	8,422,323	1,796	100%

Article bodies taking 91% is unremarkable for a content site. That line matched my expectations exactly.

The next line did not.

A generated file weighed 2.2 times all the code I wrote by hand

Narrow the scope to src and the distortion becomes obvious.

Area	Est. tokens	Files	Share of src
src/generated	289,603	2	68.9%
src/app	80,278	36	19.1%
src/components	37,332	22	8.9%
src/config	8,115	2	1.9%
src/lib and others	5,029	10	1.2%

All seventy hand-written source files together come to 130,754 tokens. src/generated reaches 289,603 across two files, with articles.json alone accounting for 284,072. A ratio of 2.21.

That file is metadata assembled mechanically from the MDX sources and rewritten on every build. Reading it adds no information whatsoever. Every fact inside it already lives under content/.

Yet in any exploration scoped to src, it sits right at the front of the queue. The filename says articles.json and the path says src, and both read as important. Files that look central by name and location are sometimes the derived ones.

Converted to input price, those two files cost about USD 0.58 per read. Reading every hand-written file costs USD 0.26. The side carrying zero new information was more than twice as expensive.

Prices move, of course, with contracts and model revisions. Replace PRICE_PER_MTOK with a figure you have checked yourself rather than trusting a number in an article.

✦

Thank you for reading this far.

Continue Reading

What follows includes implementation code, benchmarks, and practical content we hope you'll find useful. This site runs without ads — server and development costs are supported entirely by members like you. If it's been helpful, we'd be truly grateful for your support.

WHAT YOU'LL LEARN

✦You will be able to pinpoint which parts of your own repository eat context, using a single command, and decide what to exclude with evidence behind it

✦You will be able to avoid the failure where generated artifacts quietly enter exploration, inflating input cost and muddying reasoning at the same time

✦You will be able to recognise the cases where a poorly scoped search costs more than simply reading the files

✦You will know when a per-area rollup hides a single heavy file, and keep the rollup and the single-file ranking side by side

Secure payment via Stripe · Cancel anytime

✦

Unlock This Article

Get full access to the rest of this article. Buy once, read anytime. This site is ad-free — your support goes directly toward keeping it running.

Unlock all articles with Membership →

Pin the exclusions in settings.json

While measuring, I noticed something: this repository had neither a CLAUDE.md nor a .claude/settings.json. My operating rules lived in a separate workspace, and the repository itself had been left bare.

The first thing I added was a deny rule for generated output.

{
  "permissions": {
    "deny": [
      "Read(./src/generated/**)",
      "Read(./package-lock.json)"
    ]
  }
}

A deny rule gives no sign of whether it is actually working. I have not tested what happens when a pattern misses the files it was meant to cover, so after adding the setting I ask for the file once and watch the refusal happen with my own eyes. It takes thirty seconds.

I very nearly reached for .gitignore instead, then stopped. articles.json is something the CI build depends on, and it has to stay committed. Ignoring it would break production.

"I do not want this read" and "this must remain in the repository" are not in conflict. They simply belong in different files, and the right one was permissions.deny.

One caveat worth knowing: the deny rule applies to the Read tool. Call cat or head through bash and the same content still flows in. Rather than sealing that off, I added a single line to CLAUDE.md:

src/generated/ is built from the MDX sources. To check content, look under content/ instead.

The deny rule stops the accident; the sentence stops the detour from being attempted. Neither works properly alone.

Sometimes searching costs more than reading

"Grep first, read second" is the standard move. In a repository carrying a lot of prose, that standard can invert.

Four attempts against the same repository:

Approach	Matching lines	Est. tokens
`grep -rn 'premium' content/`	1,949	66,674
`grep -rln 'premium: true' content/articles/ja`	382	9,327
`grep -rn 'paywall' src/`	7	223
`grep -rn 'getArticleContent' src/`	4	123

The unfiltered search at the top returns 66,674 tokens on its own — heavier than reading all of src/components. And what comes back is a wall of near-identical lines, lower in decision value per token than the files themselves would have been.

The paywall search returned 223 tokens and pointed at two files. Reading both costs 8,447. Together, 8,670 tokens against the 420,357 that reading all of src would take. A factor of 48.

So the lever is not "grep versus Read." It is pattern selectivity. The moment your term produces three-digit match counts, grep has stopped narrowing anything and become a bulk printer.

I now insert one step before exploring:

Count first: grep -rc PATTERN TARGET | awk -F: '{s+=$2} END{print s}'
Above three digits, throw the pattern away and find a more specific identifier — a function name, a type, a config key
Once it drops to two digits or fewer, run it again with -n and actually look

When no specific identifier comes to mind, that turned out to be a useful signal in itself: I had not yet articulated what I was looking for.

File size ranking is not importance ranking

Ranking the hand-written files by weight:

File	Est. tokens	What it holds
`src/app/[locale]/about/page.tsx`	9,283	A static introduction page
`src/app/[locale]/support/SupportClient.tsx`	7,266	Support page UI
`src/config/gone-slugs.ts`	6,292	A list of removed slugs
`src/components/ui/PremiumPaywall.tsx`	4,604	The paywall itself

The top three share a trait: almost no logic. Two are prose and markup; the third is an array of strings. They are long because many things are listed, not because anything is hard.

PremiumPaywall.tsx — the answer to my original question — sits fourth. "Read the biggest files to understand the codebase" would have missed three times before landing.

Sorting by size is useful, but not for choosing what to read first. It is for finding what not to read. I now treat that ranking as an exclusion shortlist rather than a priority queue.

Counting again two months later, and what my own rollup had hidden

Two months after writing this, on October 10, I ran the same script again. The site now holds 1,776 articles across both languages, and content/articles comes to 8,611,107 tokens.

src had barely moved. articles.json is still 284,072 tokens. Hand-written source grew to 72 files and 139,416 tokens, so the ratio slipped from 2.21 to 2.08. The conclusion stands.

What caught my eye was a different line: second place in the single-file ranking was package-lock.json, at 138,805 tokens.

In August I had already put that name in the deny list. But I had done it out of habit — "lockfiles are things you don't make it read" — not because a number told me to. In the per-area table the file dissolved into a single "repository root" row, 15 files and 162,421 tokens, and nothing there suggested that 85% of it was one file.

138,805 tokens is almost exactly the weight of all my hand-written src (139,416). Read at the $2 input price I use as a placeholder, that is about $0.28 a pass. I had landed on the right answer by habit, with the evidence arriving only afterwards.

The cause was group(), which folds paths into their first two levels and sweeps every top-level file into "(root)". Per-area rollups are good for seeing the shape of things, and exactly wrong at spotting one file that sticks out.

Read the per-area table to see the shape, and the single-file ranking to find the outlier — always side by side.

I have left the script as it is. Printing both views together is what made me notice, and I would rather not tidy that away.

Three rules I adopted, and the ones I turned down

What went into practice:

Derived output stays unread. src/generated/** and package-lock.json (138,805 tokens on its own) go into permissions.deny, with a one-line reason in CLAUDE.md
Count before you search. A pattern producing three-digit hits is not narrowing anything; replace it with a more specific identifier first
Hand over a map. A short note in CLAUDE.md about where things live, so the starting point is not left to guesswork

If you only adopt one of the three, I would recommend the first. It costs a few lines of configuration and then never asks anything of you again. The other two take a while to become habit.

The rejected options may be more useful than the accepted ones.

Adding generated files to .gitignore. The CI build depends on them, so excluding them breaks production. Not wanting something read is a different problem from not wanting it stored.

Just using a wider context window. Lower input prices make "read everything, then think" viable in principle. But derived files add no information — what they add is ambiguity, the same facts arriving twice through different paths. That is a reasoning-quality problem before it is a cost problem.

Delegating exploration to a subagent. Isolation helps, certainly. But if the exploration strategy is wrong, the subagent pays the same waste behind a wall. Deciding what not to read comes first in the ordering.

The one move worth making tomorrow

Run python3 ctxweight.py src against your own repository, once. If something you have no memory of writing appears near the top, that is your first deny-rule candidate.

For me, excluding two files cut the cost of exploring src to roughly a third. I had not expected a single setting to move the number that far.

If you want to go further on the session side, Managing the Claude Code Context Window collects the implementation patterns that keep long sessions light. If the problem is that reads themselves are being truncated and your numbers no longer add up, when the Read tool cuts large files short answers that more directly.

I plan to rerun the measurement whenever the article count or the generated output grows noticeably. Ratios like these drift if nobody looks.

Thank You for Reading

Claude Lab is ad-free, supported entirely by members like you. We publish practical guides daily with implementation code, benchmarks, and production-ready patterns. If you've found it useful, we'd love to have you on board.

✦Copy-paste ready implementation code
✦New advanced guides published daily
✦$5/mo or $10 for lifetime access

View Membership →