Once I passed a dozen MCP servers, I stopped trusting the startup list

Six kinds of counterpart, six outcomes

To confirm the classification behaves, I built six servers that differ only in how they respond — instant, delayed, silent, crash-on-launch, and protocol rejection — using nothing but the standard library.

The third row is the one to read closely. It is the exact same healthy server as row two, but trimming the deadline from 5.0s to 2.0s alone flipped READY into TIMEOUT.

And in the log, that TIMEOUT is character-for-character identical to the one from the genuinely silent server in row four. A tightened deadline quietly moves healthy servers onto the broken pile.

Meanwhile rows five and six both resolve in 0.02 seconds. The two categories you can actually fix announce themselves immediately. Designing a longer wait in the hope that "more patience might reveal something" does not surface those discoveries one second sooner.

At fleet scale, concurrency mattered and the deadline did not

I grew this to fourteen servers and measured. The mix: nine healthy servers responding in 0.2s, two healthy servers taking 2.5s, and three that never answer. Deadline 5.0s across the board.

Identical verdicts, and the time dropped to under a quarter. The slowest READY on the parallel run was 2.573s, which means roughly half of the 5.09s total was spent waiting for healthy servers and the rest was spent watching three silent ones run out their deadline.

Here is where my expectation was wrong. Under parallel probing the total is the maximum across servers, so as long as even one silent server remains, the total pins to the deadline value itself. Cutting three silent servers to two, or to one, leaves 5.09s untouched.

For a while I believed that removing servers I no longer used would make startup lighter. That held while I was probing sequentially, and stopped holding the moment I went parallel. Cleanup pays off in wall time only when you eliminate every silent counterpart. Read the other way: once probing is parallel, leaving broken servers in place costs you nothing at startup.

Trimming the deadline turns healthy servers into failures

So what deadline is right? I spread healthy response times from 0.05s to 3.4s across ten servers, mixed in three silent ones, and varied only the deadline.

The shape is straightforward. Total wall time tracks the deadline almost exactly, and the READY count plateaus once you clear the slowest healthy server at 3.4s. Stretching 5.0s to 8.0s buys nothing and costs three seconds.

The interesting side is the other direction. Dropping to 1.0s does cut wall time from 5.06s to 1.06s — a real four seconds. The price of those four seconds is three healthy servers relocated to the failure pile, where they are indistinguishable from the three that are genuinely silent.

Which means the savings from failing fast are paid for entirely by healthy-but-slow servers, and have nothing to do with how many servers are actually silent. If you cannot see what is being handed over, the trade looks free when it is not. I had been choosing the deadline from how long I was willing to wait, and that is backwards. The number to choose from is how long your slowest healthy server takes; your patience is a constraint you check afterward.

In practice I leave about 1.5x headroom above the slowest healthy value — 5.0s against a 3.4s tail here. When the counterpart is an external service whose response time swings day to day, take that headroom up front, or your fleet will quietly shrink on busy mornings.

A chatty server turned a fixable failure into an unfixable one

Running the probe above across my own fleet, one server kept giving me a result I could not account for. Launched by hand from npx it came up in a couple of seconds. Under the probe it returned TIMEOUT every single time. Stretching the deadline to 8s, then 15s, changed nothing.

The cause was not on the server's side. It was in probe.py: I take stderr=subprocess.PIPE and then never read from it.

Pipes have a capacity. With no reader draining it, the writer's write blocks the moment that capacity is full. A server blocked on write never gets around to reading initialize, so from my side it looks exactly like no response. On the Linux box I measured, the default capacity was 65,536 bytes.

I built a server whose only variable is how much it writes to stderr at launch, and sent the same thing to the plain probe.py and to a version that drains stderr on a separate thread. Deadline 5.0s.

The boundary sits exactly on the pipe capacity. Up to 64 KB the plain version is fine; one step past it, a perfectly healthy server waits out the full deadline and is filed as a failure.

The worse case is a server that is actually dying. I made one that prints diagnostics at launch and then exits over a missing credential, and varied only the volume of stderr.

Same server, same cause, same one-line fix. Yet because its diagnostics run to 128 KB, an EXITED I could have fixed in a minute arrives as a TIMEOUT I can do nothing about. The line naming the reason stays inside the pipe and never reaches me.

Earlier in this article I wrote that trimming the deadline turns healthy servers into failures. The same thing was happening in a place that has nothing to do with the deadline. A classifier is only as honest as its willingness to let the other side finish speaking.

The fix is short. Put one thread right after Popen that keeps reading stderr, and hold on to the tail.

# probe.py, corrected (diff only; place immediately after Popen)
    err_tail = []
 
    def drain_stderr():
        for line in proc.stderr:          # read until the far end closes; never let the pipe fill
            err_tail.append(line)
            if len(err_tail) > 200:       # do not keep all of it; only the tail is used as detail
                del err_tail[:100]
 
    threading.Thread(target=drain_stderr, daemon=True).start()

In the EXITED branch, drop proc.stderr.read() and take the last line of err_tail instead.

    if not line:
        proc.kill()
        tail = "".join(err_tail).strip().splitlines()
        return (name, EXITED, elapsed, tail[-1] if tail else "closed stdout without responding")

Looking back, of course it came up when I launched it by hand. Attached to a terminal, stderr flows to the screen and accumulates nowhere. It only jams when a probe is holding the other end of a pipe. I may be generalising too far, but most bugs that refuse to reproduce are probably differences in the conditions I am reproducing them under — I spent the better part of two days on that one server, suspecting its config the whole time.

stdout is safe here because I read one line and then proc.kill(). That said, a server that floods stdout right after the handshake would jam for the same reason. Adding a second drain thread costs nothing, so add one if that describes your fleet.

Give each outcome its own response

Once classification works, all that remains is routing. Here is how I assign it.

TIMEOUT is the only one I allow a retry, because it is the only outcome time can resolve. The four servers that came back five minutes into that morning were all of this kind. Retrying EXITED or PROTOCOL_ERROR just buys the same failure twice.

The visibility split exists because I run some of this unattended. Working as an indie developer, where the first thirty minutes of the morning largely decide how far the day gets, narrowing the lines worth reading converts directly into time. Printing every TIMEOUT fills the log with "three down again today," and the single EXITED line hiding among them gets skimmed past. Put the fixable failures in front, fold the unfixable ones into a count. That alone cut the number of lines worth reading first thing in the morning to almost nothing.

Unattended runs need one more step: if the READY set differs from last time, decide whether to proceed at all. Whether it is acceptable to run to completion when yesterday's 14-of-14 is today's 11 depends on the nature of the job. For aggregation or delivery work — the kind that finishes cleanly even with pieces missing — treating the READY count as a precondition proved safer.

Not re-measuring on every startup

The four outcomes differ enormously in how likely they are to change. EXITED and PROTOCOL_ERROR will return the same answer until you fix the config. TIMEOUT changes with time. Giving them different lifetimes rather than one uniform cache makes subsequent startups much lighter.

# registry.py - remember results with a lifetime per outcome
import json, os, time
from concurrent.futures import ThreadPoolExecutor
from probe import probe, READY, TIMEOUT, EXITED, PROTOCOL_ERROR
 
CACHE_PATH = os.path.expanduser("~/.cache/mcp-readiness.json")
TTL = {READY: 900, TIMEOUT: 0, EXITED: 3600, PROTOCOL_ERROR: 3600}
#      ready: 15 min | no-response: always re-measure | fixed-until-you-fix-it: 1 hour
 
def _load():
    try:
        with open(CACHE_PATH) as fp:
            return json.load(fp)
    except (OSError, ValueError):
        return {}
 
def refresh(servers, timeout=5.0, max_workers=16):
    """servers: [(name, command, env_overrides), ...]"""
    cache, now, pending, results = _load(), time.time(), [], {}
 
    for name, command, env_overrides in servers:
        entry = cache.get(name)
        if entry and now - entry["at"] < TTL.get(entry["status"], 0):
            results[name] = entry          # still within its lifetime; skip the handshake
        else:
            pending.append((name, command, env_overrides, timeout))
 
    if pending:
        with ThreadPoolExecutor(max_workers=max_workers) as pool:
            for name, status, elapsed, detail in pool.map(lambda a: probe(*a), pending):
                results[name] = {"status": status, "elapsed": elapsed,
                                 "detail": detail, "at": now}
 
    os.makedirs(os.path.dirname(CACHE_PATH), exist_ok=True)
    tmp = CACHE_PATH + ".tmp"
    with open(tmp, "w") as fp:
        json.dump(results, fp, ensure_ascii=False)
    os.replace(tmp, CACHE_PATH)            # never leave a half-written JSON behind
    return results

TTL[TIMEOUT] = 0 is the point of the whole thing. No-response is re-measured every time. Cache it as long as the others and a server that failed once at 8am is treated as absent for the rest of the day. Those four servers from that morning came very close to costing me a full day of output that way.

os.replace guards the write so an interrupted run does not leave corrupt JSON behind. If that file becomes unreadable, _load() returns empty and every server gets re-probed. That fails safe, but if it happens on every startup, caching stopped meaning anything.

What I still have not settled

Whether to set the deadline per server is still open. My fleet has a 0.05s responder living next to a 3.4s one, and handing 5.0s to all of them is plainly crude. But per-server values mean per-server maintenance. I tried deriving the deadline automatically from past READY timings; it learned from busy days and the deadline kept creeping upward, so I rolled it back.

The other loose end is what happens when a server dies after being judged READY. A successful handshake is not a promise about the rest of the session. That sits in the call layer rather than the connection layer, and it needs a different mechanism.

If you want a starting point, run probe.py once across your current MCP server list with a 5.0s deadline. The moment the results split into four outcomes, you know which ones deserve your attention today. In my case, out of seven lines saying "connection failed," exactly two were worth fixing.