Make Your Nightly MCP Connectors' Health Visible — A Lightweight Ledger for Solo Operators

One morning I opened the update logs for my four sites and noticed that just one of them hadn't gained a new article. The task's result was recorded as a success. Yet nowhere could I see which connector had been slow, or which tool call had failed midway and been quietly swallowed.

It took me over an hour to track down. The truth turned out to be that one MCP connector had slowed almost to its timeout, and a retry papered over it, so the run as a whole finished as a "success." The signs of slowness had surely been showing for a while — I had simply never once looked at them.

On July 7, connector observability reached public beta for Enterprise, letting admins monitor connector adoption, errors, latency, and usage across Claude products. It's an enviable feature. But as an indie developer running scheduled tasks solo across four sites, I don't have that admin dashboard.

If you don't have it, just keep the essentials yourself. In this article we'll build, with code that actually runs, from a "health ledger" that appends a single line per tool call, up through a weekly roll-up of error rate and latency, and finally to making regressions ring a bell.

Three blind spots a "success" log can't cover

First, let's be clear on why a pass/fail result isn't enough. The three blind spots I actually walked into were these.

What they share is that none can be caught at the granularity of "a single run." Degradation shows up across many calls. That's exactly why you need to keep, line by line, not just pass/fail but "which connector, which tool, how many milliseconds, succeeded or failed." Roll it up afterward and the trend appears.

Keep just one line first — an append-only health ledger

The first step is almost disappointingly plain. Wrap a tool call in a single function and append the result as JSON Lines (one record per line). You can start by adding a few lines to an existing scheduled task.

# health-ledger.sh — record one MCP tool call as one line
LEDGER="${LEDGER:-$HOME/ops/mcp-health.jsonl}"
mkdir -p "$(dirname "$LEDGER")"
 
record_call() {
  # usage: record_call <connector> <tool> -- <command to run...>
  local connector="$1" tool="$2"; shift 2
  [ "$1" = "--" ] && shift
  local start_ms end_ms status err latency
  start_ms=$(date +%s%3N)                 # GNU date millisecond epoch
  if "$@"; then
    status="ok"; err=""
  else
    status="fail"; err="exit_$?"          # keep the failed command's exit code
  fi
  end_ms=$(date +%s%3N)
  latency=$(( end_ms - start_ms ))
  printf '{"ts":"%s","connector":"%s","tool":"%s","latency_ms":%d,"status":"%s","err":"%s"}\n' \
    "$(TZ=Asia/Tokyo date -Iseconds)" "$connector" "$tool" "$latency" "$status" "$err" \
    >> "$LEDGER"
}

The caller just wraps the usual command in record_call.

source health-ledger.sh
 
record_call github issue_list -- gh issue list --repo me/app --limit 20
record_call stripe recent_charges -- node scripts/fetch-charges.mjs
record_call notion sync_pages   -- node scripts/notion-sync.mjs

The key here is to always keep a line even on success. If you only record failures, you have no denominator and can't compute an error rate. Only with the total number of calls does "1 failure out of 10" become a ratio that means something as a trend.

date +%s%3N is GNU date's millisecond form. macOS's stock date can't use it, so if you test locally, drop to second precision (date +%s) or use gdate. Scheduled tasks run on Linux, so it works as written.

Roll it up weekly — error rate and p95 latency

Once the ledger accumulates, roll it up per connector. What you want here isn't average latency but p95. Averages smooth over the occasional slow call and hide it. What actually creates the felt slowdown is the slow calls in the top few percent.

#!/usr/bin/env python3
# rollup.py — roll the health ledger up to per-connector error rate and p95 latency
import json, sys, collections
 
def percentile(values, p):
    if not values:
        return 0
    s = sorted(values)
    k = (len(s) - 1) * p
    lo = int(k)
    hi = min(lo + 1, len(s) - 1)
    return round(s[lo] + (s[hi] - s[lo]) * (k - lo))  # linear interpolation
 
emit_json = "--json" in sys.argv
rows = collections.defaultdict(lambda: {"total": 0, "fail": 0, "lat": []})
 
for line in sys.stdin:
    line = line.strip()
    if not line:
        continue
    r = json.loads(line)
    b = rows[r["connector"]]
    b["total"] += 1
    if r["status"] != "ok":
        b["fail"] += 1
    b["lat"].append(r["latency_ms"])
 
if emit_json:
    for name, b in sorted(rows.items()):
        print(json.dumps({
            "connector": name, "calls": b["total"],
            "err_pct": round(100 * b["fail"] / b["total"], 1),
            "p50": percentile(b["lat"], 0.5),
            "p95": percentile(b["lat"], 0.95),
        }, ensure_ascii=False))
else:
    print(f'{"connector":<12}{"calls":>7}{"err%":>8}{"p50":>9}{"p95":>9}')
    for name, b in sorted(rows.items()):
        err = 100 * b["fail"] / b["total"]
        print(f'{name:<12}{b["total"]:>7}{err:>7.1f}%'
              f'{percentile(b["lat"],0.5):>7}ms{percentile(b["lat"],0.95):>7}ms')

Running it is just a pipe.

cat ~/ops/mcp-health.jsonl | python3 rollup.py

The output looks like this.

connector     calls    err%      p50      p95
github           84     1.2%    240ms    910ms
notion           42     0.0%    180ms    430ms
stripe           28     7.1%    320ms   4200ms

The moment you see these three lines, that lost morning hour disappears. stripe's error rate stands out and its p95 tops four seconds. The isolation is already done; now you only need to go look at the stripe calls. Before and after keeping a ledger, the distance to touching the problem is completely different.

Make regressions ring a bell — thresholds and baseline comparison

Reviewing a table by eye every week doesn't last. Set thresholds so that only regressions surface without a human watching, and compare against the prior week. Here we use two criteria: alert if the error rate exceeds a limit, or if p95 exceeds a set multiple of last week's.

#!/usr/bin/env python3
# regress.py — compare this week to last week's baseline and report only regressions
import json, sys
 
THRESHOLDS = {"err_pct": 5.0, "p95_ratio": 1.5}  # error rate > 5%, or p95 >= 1.5x
 
def load(path):
    with open(path) as f:
        return {r["connector"]: r
                for r in (json.loads(l) for l in f if l.strip())}
 
cur  = load(sys.argv[1])   # this week (output of rollup.py --json)
base = load(sys.argv[2])   # last week's baseline
 
alerts = []
for name, c in cur.items():
    if c["err_pct"] > THRESHOLDS["err_pct"]:
        alerts.append(f'{name}: error rate {c["err_pct"]:.1f}% '
                      f'(threshold {THRESHOLDS["err_pct"]}%)')
    b = base.get(name)
    if b and b["p95"] > 0 and c["p95"] / b["p95"] >= THRESHOLDS["p95_ratio"]:
        alerts.append(f'{name}: p95 {b["p95"]}ms->{c["p95"]}ms '
                      f'({c["p95"] / b["p95"]:.1f}x)')
 
if alerts:
    print("Regression detected:")
    for a in alerts:
        print(" -", a)
    sys.exit(1)   # non-zero exit tells the scheduled task it failed
print("Within thresholds.")

Wiring it into operations is just saving the roll-up and comparing.

# roll up this week and save as JSON
cat ~/ops/mcp-health.jsonl | python3 rollup.py --json > ~/ops/summary-cur.jsonl
 
# compare against last week's baseline (non-zero exit on regression)
python3 regress.py ~/ops/summary-cur.jsonl ~/ops/summary-base.jsonl \
  || echo "Review: connector health regressed this week"
 
# if clean, promote this week to the next baseline
cp ~/ops/summary-cur.jsonl ~/ops/summary-base.jsonl

There's a reason p95_ratio is a ratio against the prior week rather than an absolute value. Each connector has a different baseline speed, so a fixed millisecond threshold is lenient on the fast ones and harsh on the slow ones. Judging by "how much worse than that connector's own normal" reduces both false alarms and misses. This idea is close to how an unattended pipeline rejects stale input, which I discuss in Don't Let Stale Input Silently Degrade You — A Freshness Contract for Unattended Pipelines.

What to measure, and what not to

More instruments is not better. The trick to sustaining a ledger in solo operations is to keep only the metrics that lead to a decision. Here is where I currently draw the line.

A ledger has another benefit: you don't panic when something fails. If the degradation is recorded as numbers, you can confirm "since when, and which connector" as fact before trying to guess at causes. For how to catch a silent success, reading Recorded as a Success but Zero Output — Stopping Silent Failures in Cowork Scheduled Tasks with an End-of-Run Assertion alongside this gives you both wheels: pass/fail and health.

You need no new tools to start. Tonight, wrap just one of your scheduled task's tool calls in record_call, and tomorrow morning, confirm that one line has been added to the ledger. That's enough to begin. I started with a single connector and a single line myself. Only once it became visible did I quietly begin to understand what to fix next.

A single ledger buys back the morning hour you were about to lose. Thanks for spending this stretch of it with me.