I Preloaded a Scheduled Task with head -150: 15.6 KB Read, 3.3 KB for Today

A small reader that cuts by date block

This is the replacement. It prints the newest day in full, only the chosen sections of older days, and writes the byte count to stderr.

#!/usr/bin/env python3
"""Read daily notes by date block.
Newest day in full; older days only the listed sections (A-G).
Usage: refpeek.py <file> [past_days=2] [sections=AE] [byte_ceiling=8000]"""
import re
import sys
 
 
def load_blocks(path):
    blocks, cur = [], None
    with open(path, encoding="utf-8") as f:
        for line in f.read().split("\n"):
            m = re.match(r"## (\d{4}-\d{2}-\d{2})", line)
            if m:
                cur = {"date": m.group(1), "lines": [line]}
                blocks.append(cur)
            elif cur is not None:
                cur["lines"].append(line)
    return blocks
 
 
def pick_sections(lines, keep):
    out, on = [lines[0]], True
    for line in lines[1:]:
        m = re.match(r"### ([A-G])\.", line)
        if m:
            on = m.group(1) in keep
        if on:
            out.append(line)
    return out
 
 
def main(path, past_days=2, keep="AE", budget=8000):
    blocks = load_blocks(path)
    if not blocks:
        sys.exit("FATAL: zero '## YYYY-MM-DD' headings. The format may have changed")
    out = list(blocks[0]["lines"])
    for b in blocks[1:1 + past_days]:
        out += pick_sections(b["lines"], set(keep))
    text = "\n".join(out)
    size = len(text.encode("utf-8"))
    note = "" if size <= budget else f" over ceiling {budget}"
    sys.stderr.write(f"blocks={1 + min(past_days, len(blocks) - 1)} bytes={size}{note}\n")
    print(text)
 
 
if __name__ == "__main__":
    a = sys.argv
    main(a[1],
         int(a[2]) if len(a) > 2 else 2,
         a[3] if len(a) > 3 else "AE",
         int(a[4]) if len(a) > 4 else 8000)

In the task's instructions, the single head -150 daily_notes.md line became:

python3 "$WS/_tools/refpeek.py" "$WS/daily_notes.md" 2 AE 8000

A line such as blocks=3 bytes=5664 lands on stderr, so the run log shows how many bytes that morning's preload read. Going over the ceiling does not stop the output; it only appends "over ceiling". A preload that is cut short is harder to debug than one that read a bit too much and said so.

How the bytes moved

Same file, different numbers of past days and sections.

Keeping only A (primary sources) and E (deadlines) from older days gives 5,664 bytes for three days, 36% of the old read. Sources are useful for catching corrections; deadlines carry across days.

Note that this table shows bytes read, not whether today's decisions were just as good. That is why the next check was needed.

The counterintuitive part: cutting by date also cuts unused ideas

When I removed B (pain points) from older days, I assumed old B sections were already used up.

I looked at B for the past three days. The 10-17 and 10-16 sections were just "none today" notes, about 300 and 420 bytes. But 10-15 held four candidate ideas not yet turned into anything, 1,698 bytes.

With past_days=2, the whole 10-15 block is out of range. Drawing the line by date made those four ideas vanish from the preload. I also searched the names of what I produced afterwards for words matching those four; that is only a word match and not proof, but I found nothing.

"Drop old days" is not the same as "drop what is used up". The file is stacked by date, but an idea's lifetime has nothing to do with dates.

For now I split it like this:

• Sections refreshed or valid across days (A, E) can be cut by date.
• Idea sections (B, G) are cut by state: used ideas get a mark, and only unmarked ones are read for a few days.
• The mark is just [used] appended to the idea's line, because other tasks read the same file and I do not want to change its format much.

The state-based reader for idea sections is still a prototype. Whether hand-marking [used] survives in practice is something I will decide after a few weeks.

Pitfalls

A changed heading format makes the reader read nothing, silently. If the date heading turns into ## 2026/10/18 or # 2026-10-18, the regex finds zero blocks. The reader above calls sys.exit on zero; my first version printed nothing and ended, and the task reported success on an empty preload. Treat empty output as a failure.

Do not mix stderr into stdout. If the byte count goes through print, that line lands inside the preload text. Write log lines to sys.stderr and keep a fixed prefix (blocks=) so you can search for it later.

Section letters are hard-coded to A-G. If an ### H. appears, [A-G] will not match it and it will be treated as a continuation of the previous section. On the day a section is added, update the regex the same day; I wrote that into the task's instructions.

The byte ceiling is yours to choose. 8,000 bytes came from my file, where "newest + 3 past days (A, E)" was 7,526 bytes. Run blockstat.py on your own file first and decide from its shape.

What to try next

First, measure what your head -N or tail -N preload actually carries on your own file, with a short script like blockstat.py. If date blocks differ by nearly 2x in bytes, the line limit is not standing in for a budget.

Then, before dropping old days, check whether any ideas nobody has used yet are still sitting in them. Had I not checked, I would have let go of four ideas without knowing it.

This is a sequel to counting how many times a runbook is re-read before trimming it. That article asks how often the preload is re-read; this one asks what gets read the first time. Together they make it easier to see where trimming is worth it.

Like the Japanese-filename problem where the voiced marks come apart, a skew you only see once you count turned out to be hiding in the preload as well. I intend to keep the order: put numbers out first, then cut.