The 400-Character Preview That Was Still Holding On to a Megabyte

The tricks that look like they copy, but do not

So which rewrites actually detach the parent? I lined up every idea I had and measured them under identical conditions — 4 MB parents, twenty iterations, 200-character previews.

// exp2b.mjs — one technique per process, so leftovers cannot contaminate the next run
const F = {
  'slice':        s => s.slice(0, KEEP),
  'substring':    s => s.substring(0, KEEP),
  'template':     s => `${s.slice(0, KEEP)}`,
  'String()':     s => String(s.slice(0, KEEP)),
  'concat-empty': s => '' + s.slice(0, KEEP),
  'space-slice1': s => (' ' + s.slice(0, KEEP)).slice(1),
  'split-join':   s => s.slice(0, KEEP).split('').join(''),
  'buffer':       s => Buffer.from(s.slice(0, KEEP), 'utf8').toString('utf8'),
  'json':         s => JSON.parse(JSON.stringify(s.slice(0, KEEP))),
  'array-join':   s => Array.from(s.slice(0, KEEP)).join(''),
  'repeat1':      s => s.slice(0, KEEP).repeat(1),
  'padEnd':       s => s.slice(0, KEEP).padEnd(KEEP),
  'normalize':    s => s.slice(0, KEEP).normalize(),
};

I had put my money on the template literal and on String(). Both read like "construct a fresh string." Both sail straight through.

So do '' + s, repeat(1), and padEnd called with the string's existing length. From the engine's point of view, returning the input unchanged is the correct answer whenever the output would be identical. The copy we were counting on as a side effect is precisely what the optimizer exists to remove.

What works is only the rewrites that force the internal representation to be rebuilt: prepend a character and strip it, explode into an array and rejoin, round-trip through bytes or JSON. Every one of them looks like pointless work, and every one of them is the first thing a reviewer would delete.

The fastest fix is the one that looks most wasteful

I measured the cost too: 200 characters extracted from a 256 KB source, ten thousand times, median of seven runs.

The scrappiest-looking option, (' ' + x).slice(1), is the fastest of the working set — roughly 2.4 times quicker than the Buffer round-trip.

Meanwhile the two techniques that feel the most thorough, going through a character array, are more than ten times slower than that. Reasonable in hindsight, but I did not expect confidence and speed to sort in opposite directions.

The catch with (' ' + x).slice(1) is that nothing in the code explains why it exists. My compromise: Buffer round-trip wherever the cost is invisible, (' ' + x).slice(1) only on hot paths, and a comment either way.

Strings that cross a serialization boundary leave their parents behind

After finishing the comparison table I realized there was one API I had forgotten to try: structuredClone. It is the standard deep-copy primitive, so if a parent could survive that, we would be in genuinely unpleasant territory.

kept.push(structuredClone(full.slice(0, KEEP)));

The result was 4.00 MB — against 80.00 MB for the bare slice, the parents are cleanly detached. Wrapping the preview in an object and passing it through as structuredClone({ preview: p }) measured the same 4.00 MB.

The reason makes sense once you think about the implementation. structuredClone serializes the value and reconstructs it, so every string gets written out as bytes exactly once. The reconstructed side is a flat, freshly allocated string with no way to reference a parent.

The same logic should apply to postMessage in worker_threads, so I measured that too. Twenty 200-character previews, each sliced from a 4 MB parent, sent to a worker as sliced strings.

// sender — pass the sliced strings as-is
const w = new Worker(new URL(import.meta.url));
w.postMessage(previews);
 
// worker — hold the received array and measure the delta
parentPort.on('message', (previews) => {
  held = previews;
  global.gc();
  parentPort.postMessage(
    `heapDelta = ${((process.memoryUsage().heapUsed - base) / 1048576).toFixed(2)} MB`);
});

The receiver's heapUsed delta was 0.01 MB. The same twenty strings that pinned 80 MB on the sending side are just 4,000 characters once they cross the boundary.

So if your architecture ships history to a worker or a separate process for aggregation, the receiving side never sees this problem. The flip side is an asymmetry that can mislead you: the worker measures clean while only the sending process bloats. I did not think of that asymmetry at first, and burned time staring at snapshots taken on the wrong side.

Should you then just delegate flattening to structuredClone? Not quite. Under the same conditions as the earlier benchmark (200 characters out of 256 KB, 10,000 iterations, median of 7 runs) it cost 1.08 µs per call. The baselines within that same run were 0.11 µs for bare slice and 0.63 µs for the Buffer round trip (a separate run from the earlier table, so the absolute values drift slightly). That puts it at roughly 1.7x the Buffer round trip and about 10x a bare slice.

My rule of thumb: treat serialization-driven flattening as something you get for free when you were crossing a boundary anyway, and reach for the Buffer round trip or (' ' + x).slice(1) when flattening itself is the goal.

Putting it in a tool wrapper and measuring again

Everything so far is synthetic. Here is the same question shaped like a real wrapper: a tool that returns 1 MB, called 200 times, with a 400-character preview kept in history.

// exp10.mjs — node --expose-gc exp10.mjs [naive|flatten]
const FLATTEN = process.argv[2] === 'flatten';
const CALLS = 200;
const OUT = 1024 * 1024;
const KEEP = 400;
 
function truncatePreview(s, keep) {
  if (s.length <= keep) return s;
  const head = s.slice(0, keep);
  // head still references the whole parent; the round-trip cuts it loose.
  return FLATTEN ? Buffer.from(head, 'utf8').toString('utf8') : head;
}
 
async function callTool(i) {
  return 'L'.repeat(OUT) + '\n#' + i;   // stand-in for a log-reading tool
}
 
const history = [];
global.gc();
const t0 = Date.now();
 
for (let i = 0; i < CALLS; i++) {
  const out = await callTool(i);
  history.push({ tool: 'read_log', preview: truncatePreview(out, KEEP), fullBytes: out.length });
}
 
global.gc();
const m = process.memoryUsage();
console.log(FLATTEN ? 'flatten' : 'naive  ',
            'heapUsed =', (m.heapUsed / 1048576).toFixed(1), 'MB',
            '/ rss =', (m.rss / 1048576).toFixed(1), 'MB',
            '/ elapsed =', Date.now() - t0, 'ms');

Four runs each, with almost no variance.

Identical information retained. About 44 times less heap, about 5.6 times less RSS.

The bill is 16 milliseconds across 200 calls — 0.08 ms each. Against the latency of the tool itself, that is noise.

Keeping fullBytes separately is deliberate. Throwing away the text while remembering how large it was lets you judge later whether the truncation was reasonable, and it makes it far less likely that the model reads a preview and assumes it is the whole thing.

Turning suspicion into evidence with a heap snapshot

"Memory is growing" is not the same as "sliced strings are the cause." Being able to count the retained parents turns the fix into something you can verify before and after.

Node ships v8.writeHeapSnapshot(), and the .heapsnapshot it produces is JSON. No external tooling required.

Snapshot nodes have a type of sliced string, and an internal edge from each one points at its parent. Follow that edge, filter to parents above a size floor, and sum.

#!/usr/bin/env node
// sliced_audit.mjs
// Count sliced strings in a heap snapshot and the parents they keep alive.
// Usage: node sliced_audit.mjs <snapshot> [min parent bytes] [budget MB]
import fs from 'node:fs';
 
const file = process.argv[2];
const PARENT_MIN_BYTES = Number(process.argv[3] || 262_144); // only parents >= 256 KB
const BUDGET_MB = Number(process.argv[4] || 8);
 
const snap = JSON.parse(fs.readFileSync(file, 'utf8'));
const { node_fields: nf, node_types: nt, edge_fields: ef, edge_types: et } = snap.snapshot.meta;
const NS = nf.length, ES = ef.length;
const TYPE = nf.indexOf('type'), NAME = nf.indexOf('name'),
      SIZE = nf.indexOf('self_size'), ECOUNT = nf.indexOf('edge_count');
const ETYPE = ef.indexOf('type'), ETO = ef.indexOf('to_node');
const nodeTypes = nt[0], edgeTypes = et[0];
 
// Precompute node index -> first edge index. Counting from the top each time is O(n^2).
const edgeStart = new Uint32Array(snap.nodes.length / NS);
for (let i = 0, o = 0, e = 0; o < snap.nodes.length; o += NS, i++) {
  edgeStart[i] = e;
  e += snap.nodes[o + ECOUNT];
}
 
// A Map keyed by parent offset prevents double counting when several slices share a parent.
const parents = new Map();
let sliced = 0;
 
for (let i = 0, o = 0; o < snap.nodes.length; o += NS, i++) {
  if (nodeTypes[snap.nodes[o + TYPE]] !== 'sliced string') continue;
  sliced++;
  const base = edgeStart[i] * ES;
  for (let k = 0; k < snap.nodes[o + ECOUNT]; k++) {
    const eo = base + k * ES;
    if (edgeTypes[snap.edges[eo + ETYPE]] !== 'internal') continue;
    const po = snap.edges[eo + ETO];
    const bytes = snap.nodes[po + SIZE];
    if (bytes >= PARENT_MIN_BYTES) parents.set(po, bytes);
  }
}
 
let retained = 0;
const samples = [];
for (const [po, bytes] of parents) {
  retained += bytes;
  if (samples.length < 3) {
    samples.push({
      parentMB: +(bytes / 1048576).toFixed(2),
      head: (snap.strings[snap.nodes[po + NAME]] || '').slice(0, 32),
    });
  }
}
 
const retainedMB = +(retained / 1048576).toFixed(2);
console.log(JSON.stringify({
  slicedStrings: sliced, retainedParents: parents.size, retainedMB, budgetMB: BUDGET_MB, samples,
}));
 
process.exit(retainedMB > BUDGET_MB ? 1 : 0);

The edgeStart table is the part worth keeping. Snapshot edges live in one flat array for the whole heap, so finding a given node's edges means summing every preceding node's edge count. Written naively that is a nested loop, and past a few tens of thousands of nodes the audit itself starts taking tens of seconds.

Run against a process that stacked fifty 2 MB outputs:

$ node sliced_audit.mjs naive.heapsnapshot
{"slicedStrings":54,"retainedParents":50,"retainedMB":100,"budgetMB":8,
 "samples":[{"parentMB":2,"head":"LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL"}, ...]}
exit=1

$ node sliced_audit.mjs flatten.heapsnapshot
{"slicedStrings":5,"retainedParents":1,"retainedMB":2,"budgetMB":8,"samples":[...]}
exit=0

Fifty parents, 100 MB, right there in the output. The flattened build shows one parent and 2 MB. The head field gives you the first characters of what is being retained, which is usually enough to identify the offending tool.

The audit took 0.09–0.10 seconds against a 5 MB snapshot. Cheap enough for CI.

One caveat: slicedStrings will never be zero. Libraries slice short strings constantly and that is entirely fine. The number that carries signal is the summed size of parents above your floor, not the count.

A red audit does not have to mean a restart

When the audit script exits 1, the first remedy that comes to mind is a restart. But a long-lived process usually has sessions in flight, and you cannot always afford to drop them.

So I measured a way to make an already-bloated process let go of its parents while it keeps running. The repair is nothing clever: walk the history and overwrite each preview with a flattened copy.

for (const h of history) {
  h.preview = Buffer.from(h.preview, 'utf8').toString('utf8');
}

Here is the before and after on a process that had accumulated 200 naive calls of 1 MB each.

All 200 parents were released at once, and the replacement loop took 0.85 ms for 200 entries. The process returns essentially to its pre-leak footprint without ever going down.

The easy thing to miss is the obvious one: a parent is freed only when its last reference disappears. Creating a flattened copy is not enough — every variable and property that pointed at the sliced string has to be overwritten with that copy. If a cache or a log buffer holds the same preview elsewhere, the parent stays pinned until you replace that reference too.

I wired this repair into the same signal handler as the audit. On SIGUSR2: snapshot, audit, repair if over budget, audit again. If retainedMB drops on the second pass, that doubles as confirmation that sliced strings were the culprit. If it does not drop, something else is still holding on.

In Python the trap sits one step to the side

I ran the same shape in Python, and the answer flips. Slicing a str genuinely copies.

import tracemalloc
SIZE, N, KEEP = 4 * 1024 * 1024, 20, 200
tracemalloc.start()
base = tracemalloc.get_traced_memory()[0]
kept = [('x' * SIZE + '#' + str(i))[:KEEP] for i in range(N)]
cur = tracemalloc.get_traced_memory()[0]
print(f"retained = {(cur - base) / 1048576:.2f} MB")

Python 3.10.12 reports retained = 4.00 MB — one parent's worth, with no accumulation.

Swap in a memoryview over bytes and the trap reappears.

kept.append(memoryview(full)[:KEEP])   # full is bytes

That one reports retained = 80.01 MB. A memoryview is a zero-copy view by definition, so this is the documented behavior rather than a surprise — but it is the same failure mode wearing different clothes.

The general lesson is that this is not a Node defect. Every language draws the line between "slicing copies" and "slicing views" somewhere, and it is worth knowing where yours draws it before you store the result.

Anthropic shipped a fix for exactly this shape of bug on their side, where the untruncated text of a truncated MCP tool result stayed in memory for the life of the session. If the official implementation could hold on to it, there is no reason to assume your own wrapper does not.

What to do before you leave a process running

Three habits are all I keep now.

1. Funnel every truncation through one function. Scattering the flattening across individual slice calls guarantees you will miss one. Have a single truncateForHistory() and forbid pushing strings into history anywhere else.
2. Wire writeHeapSnapshot() into long-lived processes. Dumping a snapshot on a signal is enough. Being able to grab the artifact the moment things look wrong is what turns diagnosis from guesswork into measurement.
3. Run the audit with a budget in CI. The script above exits 1 when the total crosses your ceiling. Expressing the threshold in bytes rather than in counts turned out to be the practical choice. And when the audit does cross the budget, a restart is not your only option — replacing the references with flattened copies reclaims the memory in place, as measured above.

And no, you should not flatten every slice. If the substring is consumed and discarded immediately, flattening is pure overhead. It only earns its keep when the result goes into something long-lived — history, caches, metric labels. Going back through the code I write as an indie developer, those three covered every real case.

If you have an MCP server or an agent sitting resident right now, start by driving a few hundred calls through it under node --expose-gc and watching heapUsed. If it climbs, the truncation line is the first place I would look.