Running Four Sites From One Managed Agent Definition

The second trap: arrays replace, they don't merge

My base agent allows several tools. For one session, I wanted to swap one of them for something else. So I passed just that one tool in tools.

The rest of the tools vanished for that session.

An array handed to sessions.update() replaces that field wholesale. It is not merged element by element. Pass one entry and that session has exactly one tool.

Put that way it's obvious. But if you read "override" as "add a delta," you end up with a nightly batch running on a shrunken tool set. I only noticed by reading the logs.

The principle I took from it is a single line. What you pass in an override is always the final state of that field. If you want to add or remove, read the base value, compose the result yourself, and pass that. Composing it safely is what the factory below is for.

A startup factory with validation

I put the intent and the safety net in one place — the code that starts a session. TypeScript:

import Anthropic from "@anthropic-ai/sdk";
 
const client = new Anthropic();
 
// The ceiling of what the base is allowed to use. Overrides beyond this are rejected.
const TOOL_ALLOWLIST = new Set([
  "agent_toolset_20260401",
  "mcp_toolset",
]);
const MCP_ALLOWLIST = new Set([
  "mcp-claudelab",
  "mcp-gemilab",
  "mcp-antigravitylab",
  "mcp-rorklab",
]);
 
type SiteProfile = {
  site: string;
  agentId: string;        // one base agent; site differences ride on the version
  agentVersion?: number;  // omit to follow the latest version
  environmentId: string;
  toolTypes: string[];    // the final state for this session
  mcpServers: { type: "url"; name: string; url: string }[];
};
 
function assertSubset(label: string, given: string[], allow: Set<string>) {
  const stray = given.filter((v) => !allow.has(v));
  if (stray.length > 0) {
    // Catch unintended privilege expansion before the session ever starts
    throw new Error(`[guard] ${label} contains items not allowed: ${stray.join(", ")}`);
  }
}
 
export async function startSession(p: SiteProfile) {
  assertSubset("toolTypes", p.toolTypes, TOOL_ALLOWLIST);
  assertSubset("mcpServers", p.mcpServers.map((m) => m.name), MCP_ALLOWLIST);
 
  if (p.toolTypes.length === 0) {
    // An empty array means "no tools". I want that to be loud, not silent.
    console.warn(`[guard] ${p.site}: toolTypes is empty — confirm this is intentional`);
  }
 
  // 1. The session only *points* at an agent. No model or system prompt here.
  const session = await client.beta.sessions.create({
    agent: p.agentVersion
      ? { type: "agent", id: p.agentId, version: p.agentVersion }
      : p.agentId,
    environment_id: p.environmentId,
    title: `${p.site} nightly`,
    metadata: { site: p.site },
  });
 
  // 2. Tools and MCP servers can be overridden session-locally after startup
  //    (only while the session is idle; arrays are full replacements)
  await client.beta.sessions.update(session.id, {
    agent: {
      tools: p.toolTypes.map((type) => ({ type })),
      mcp_servers: p.mcpServers,
    },
  });
 
  // One line recording what was swapped, so behavior can be traced later
  console.info(
    `[session] site=${p.site} agent=${p.agentId}@${p.agentVersion ?? "(latest)"} ` +
      `tools=${p.toolTypes.length} mcp=${p.mcpServers.map((m) => m.name).join("+")}`
  );
 
  return session;
}

Three things matter here.

First, assertSubset checks the ceiling before startup. Because the override is a full replacement, one stray MCP name gives that session an endpoint nobody intended. Anything outside the allowed set fails immediately.

Second, an empty array warns instead of passing quietly. "No tools" is a legitimate configuration, but in my case it was almost always a mistake. Refusing to let it through silently has saved me several mornings.

Third, every swap leaves a log line. A session-local override disappears with the session, so once it's gone there is no other record of how that batch ran.

Site differences, collapsed into a few table rows

With the factory in place, per-site differences become short and readable:

const SITE_PROFILES: Record<string, SiteProfile> = {
  claudelab: {
    site: "claudelab",
    agentId: "agent_blog_base",
    // version omitted = follows the latest
    environmentId: "env_blog",
    toolTypes: ["agent_toolset_20260401", "mcp_toolset"],
    mcpServers: [
      { type: "url", name: "mcp-claudelab", url: "https://example.internal/cl/sse" },
    ],
  },
  rorklab: {
    site: "rorklab",
    agentId: "agent_blog_base",
    agentVersion: 1772585501101368014, // pinned while I evaluate a prompt change
    environmentId: "env_blog",
    toolTypes: ["agent_toolset_20260401", "mcp_toolset"],
    mcpServers: [
      { type: "url", name: "mcp-rorklab", url: "https://example.internal/rl/sse" },
    ],
  },
};
 
async function runFor(site: string) {
  const profile = SITE_PROFILES[site];
  if (!profile) throw new Error(`Unknown site: ${site}`);
  return startSession(profile);
}

There is exactly one agent_blog_base. Shared behavior — banned words, the baseline tone rules — is written there once. What varies per site now lives in those few rows.

Onboarding a new site means adding one entry. No more duplicating a definition file and then nursing the shared parts back into sync. Collapsing four copies into one made touching the shared config feel noticeably lighter.

One caveat: if you genuinely need a different system prompt per site, this layer isn't enough. system belongs to the agent, so you either stand up a separate agent or write the shared prompt so it reads the site name from metadata. I chose the latter. The moment I had four prompts again, I'd be back where I started.

To add or remove, read the base first, then build the final state

"Always pass the final state" is correct, but in practice it means retyping the whole list every time you want to add one thing. Retyping drifts.

So I put a small helper in front of the factory that reads the current base and applies a delta.

type Delta = { add?: string[]; remove?: string[] };
 
// Apply add/remove to the current base and return the final state plus the diff
function applyDelta(current: string[], delta: Delta, label = "field") {
  const remove = new Set(delta.remove ?? []);
 
  // Catch a stale remove: the base renamed something, your config didn't
  const ghosts = [...remove].filter((n) => !current.includes(n));
  if (ghosts.length > 0) {
    console.warn(`[delta] ${label}: remove targets not present in base: ${ghosts.join(", ")}`);
  }
 
  const kept = current.filter((n) => !remove.has(n));
  const appended = (delta.add ?? []).filter((n) => !kept.includes(n));
  const next = [...kept, ...appended];
 
  return { next, diff: diffOf(current, next) };
}
 
function diffOf(before: string[], after: string[]): string[] {
  const b = new Set(before);
  const a = new Set(after);
  return [
    ...before.filter((n) => !a.has(n)).map((n) => `-${n}`),
    ...after.filter((n) => !b.has(n)).map((n) => `+${n}`),
  ];
}

On the calling side I read the base once at process start and reuse it. Re-reading per session just slows the nightly batch down.

// Fetch the base once, at startup
const base = await client.beta.agents.retrieve("agent_blog_base");
const baseTools = (base.tools ?? []).map((t) => t.type);
 
const { next, diff } = applyDelta(baseTools, { add: ["mcp_toolset"] }, "tools");
console.info(`[delta] tools ${diff.join(" ") || "(no change)"} -> ${next.join(", ")}`);
 
// Feed `next` into SiteProfile.toolTypes, then hand it to startSession

With a base of ["web_search", "text_editor"], here are five representative cases run on Node.js 22:

add bash for verification -> [web_search, text_editor, bash] diff=[+bash]
drop search               -> [text_editor] diff=[-web_search]
swap one for another      -> [text_editor, bash] diff=[-web_search +bash]
add something already there -> [web_search, text_editor] diff=[]
[delta] tools: remove targets not present in base: computer
remove something absent   -> [web_search, text_editor] diff=[]

Line four shows that adding an existing item doesn't duplicate it. The function is idempotent, so it's safe to run over machine-generated config as often as you like.

The last case is the one that earns its keep. If the base renamed a tool and your remove still names the old one, the diff comes back empty and nothing happens — meaning a tool you believed you had removed keeps running. That's what the warning is for. It goes to stderr, which is why it appears above the result line; I've left the output exactly as it printed.

And always log the diff. An override only shows you the result, so without a record of what was supposed to change, there's nothing to review.

What running it taught me

A few weeks in production settled a few things.

A session-local override stays inside that session, which is precisely why you need bookkeeping. You can list the base definition, but the base tells you nothing about how any individual session actually ran. I aggregate the log line above daily and keep a record of which site ran on which version, how many times. If you allow swapping, you own the record of the swaps. Without it, neither cost attribution nor root-cause work is possible.

Following the latest version is convenient, and it's also unintended coupling. Pass a bare ID string and the session takes whatever the latest version is at creation time. Update the base and every site moves at once. Handy — but when you want one site held back for evaluation, you have to pin it explicitly with { type: "agent", id, version }. I split my table into "may follow" and "must be pinned," and the pinned side always carries an agentVersion.

Granularity settles once you decide it by rate of change. Things that change often — connectors, which tools are permitted — go to the session side. Things that rarely change — banned words, the fundamental output shape — stay on the base. Drawing the line by rate of change stopped me hesitating over where a fix belongs. When frequently-edited details live on the base, touching shared config starts to feel risky, and that hesitation is its own cost.

Daily bookkeeping is one log file and one script

That record I mentioned isn't elaborate. It's a small script that folds the [session] lines by date, site, and version.

It assumes each line starts with a date. If your runner doesn't add one, prepend new Date().toISOString() inside the console.info.

import { readFileSync } from "node:fs";
 
// Pick out [session] lines. The leading date may come from the runner or the line itself.
const LINE =
  /^(?<ts>\d{4}-\d{2}-\d{2})\S*\s+\[session\]\s+site=(?<site>\S+)\s+agent=(?<agent>\S+)\s+tools=(?<tools>\d+)\s+mcp=(?<mcp>\S+)/;
 
const rows = readFileSync(process.argv[2], "utf8")
  .split("\n")
  .map((l) => l.match(LINE)?.groups)
  .filter(Boolean);
 
const agg = new Map();
for (const r of rows) {
  const key = `${r.ts}\t${r.site}\t${r.agent}`;
  const cur = agg.get(key) ?? { runs: 0, tools: new Set(), mcp: new Set() };
  cur.runs++;
  cur.tools.add(Number(r.tools));            // a wobble in the count stays visible
  r.mcp.split("+").forEach((m) => cur.mcp.add(m));
  agg.set(key, cur);
}
 
console.log(["date", "site", "agent", "runs", "tools", "mcp"].join("\t"));
for (const [key, v] of [...agg].sort()) {
  const tools = [...v.tools].sort((a, b) => a - b).join("/");
  console.log([key, v.runs, tools, [...v.mcp].join(",")].join("\t"));
}
 
// Runs that followed the latest version = the blast radius of your next base update
const latest = rows.filter((r) => r.agent.endsWith("@(latest)")).length;
console.log(`\nRuns following the latest version: ${latest} / ${rows.length}`);

I can't publish the real logs, so here are two days written in the same format and run through it:

date        site        agent                          runs  tools  mcp
2026-09-01  claudelab   agent_blog_base@(latest)       2     1/2    mcp-claudelab
2026-09-01  gemilab     agent_blog_base@(latest)       1     2      mcp-gemilab
2026-09-01  rorklab     agent_blog_base@17725855011…   1     3      mcp-rorklab
2026-09-02  claudelab   agent_blog_base@(latest)       1     2      mcp-claudelab
2026-09-02  gemilab     agent_blog_base@(latest)       1     2      mcp-gemilab,mcp-shared
2026-09-02  rorklab     agent_blog_base@17725855011…   1     3      mcp-rorklab
 
Runs following the latest version: 5 / 7

Look at the tools column on the first row. 1/2 means the same site ran with two different tool counts on the same day. That wobble is exactly how I first noticed a dropped tool. Override accidents don't raise exceptions, so a column that should be uniform and isn't may be the only place they ever surface.

The mcp column does the same job. mcp-shared appearing for gemilab on day two was me adding a connector. If it's intentional, the change is on the record. If it isn't, you catch it that morning.

The last line is what I check before updating the base. Five of seven runs follow the latest, so updating the base moves five runs' worth of behavior and cost the instant I do it. Knowing that number is what stopped base updates from making me nervous.

Narrowing permissions mid-conversation

One assumption underlies everything above: tool and MCP overrides only go through while the session is idle.

Send sessions.update() while the agent is running and it won't be accepted. You have to interrupt, let it settle to idle, and then send. When you're halfway through a long investigation and decide search is no longer needed, that extra step is the price.

The content of the swap follows the same rule as at startup: always the final state, never a delta. So it goes through the same validation.

// The final tool set for each phase
const PHASE_TOOLS: Record<"research" | "edit" | "verify", string[]> = {
  research: ["agent_toolset_20260401", "mcp_toolset"],
  edit: ["agent_toolset_20260401"],
  verify: ["agent_toolset_20260401"],
};
 
// Switch to the next phase. Boundary checks are shared with startup.
async function switchPhase(sessionId: string, phase: keyof typeof PHASE_TOOLS) {
  const types = PHASE_TOOLS[phase];
  assertSubset("toolTypes", types, TOOL_ALLOWLIST);
 
  const current = await client.beta.sessions.retrieve(sessionId);
  if (current.status !== "idle") {
    // Sending while running won't take. Interrupt, then call again.
    throw new Error(`[phase] ${sessionId} is ${current.status} — wait for idle`);
  }
 
  await client.beta.sessions.update(sessionId, {
    agent: { tools: types.map((type) => ({ type })) },
  });
 
  // Record when permissions changed, so it can be traced later
  console.info(`[phase] ${sessionId} -> ${phase} tools=${types.join(",")}`);
}

Research gets the MCP toolset; once drafting starts, it comes off; verification runs on the narrowest set. The interrupt is a small cost, and phase-by-phase narrowing is finally straightforward to write.

That said, my unattended nightly batches still do their swap once, right at startup. The less a configuration moves while nobody is watching, the easier failures are to trace. Mid-conversation switching is something I only use for long investigations at my own desk.

Three things that didn't work

I've mostly described what went well, so here is what didn't.

I kept the allowlist only in code, and let it drift from the base. With TOOL_ALLOWLIST hard-coded, I edited the base from the console and removed one tool. The code-side set stayed stale. Validation passed while sessions were being created against a tool the base no longer had. Now I fetch the base at startup, diff it against the allowlist, and warn on divergence. If you're going to hold a boundary in two places, the design has to include checking that the two agree.

I passed empty arrays for fields I hadn't decided yet. While one site's configuration was still unsettled, I left tools as []. An empty array means absent, not inherited. Several runs went through without the tooling my quality gate depends on. Undecided now means omitting the field entirely so it inherits; an empty array is reserved for "I really do want none of this." That distinction lives in a comment now.

I forgot to revisit a version I had pinned. I pinned one site for evaluation, then kept updating the base — and that site alone kept running on the old version. Inheritance would have carried it along automatically. Pinning is useful, and the moment you pin, you take on the duty to revisit. Now anything pinned carries a comment saying why and when to look again. It doubles as a filter: a pin you can't justify in writing was probably meant to be inheritance.

All three come down to the same thing. Overrides take effect quietly, so being wrong doesn't throw. That's exactly why the effort of adding your own warnings and logs pays for itself.

How I'd decide

Before introducing any of this, I settle three questions:

1. Which fields stay on the base and which move to the session (drawn by rate of change)
2. Where the ceiling sits for tools and MCP servers, and what gets rejected at that boundary
3. Which log holds the per-session configuration, and how it gets reviewed daily

Answering those first is what keeps permission accidents from happening.

If you have two or three genuinely different bases with large differences between them, don't force them together. The cost of validating the swaps will exceed the cost of syncing.

If the base is effectively one thing and the differences fit into "which connectors, which tools," this design earns its place. My four sites were exactly that shape.

Start small: pick one agent, hand sessions.update() a final tool set, and log the resulting diff on one line. That's where I started too.

If you're carrying several near-identical agents, I hope this gives you a way to fold them down. Thank you for reading.