The 400-Character Preview That Was Still Holding On to a Megabyte

The tricks that look like they copy, but do not

So which rewrites actually detach the parent? I lined up every idea I had and measured them under identical conditions — 4 MB parents, twenty iterations, 200-character previews.

// exp2b.mjs — one technique per process, so leftovers cannot contaminate the next run
const F = {
  'slice':        s => s.slice(0, KEEP),
  'substring':    s => s.substring(0, KEEP),
  'template':     s => `${s.slice(0, KEEP)}`,
  'String()':     s => String(s.slice(0, KEEP)),
  'concat-empty': s => '' + s.slice(0, KEEP),
  'space-slice1': s => (' ' + s.slice(0, KEEP)).slice(1),
  'split-join':   s => s.slice(0, KEEP).split('').join(''),
  'buffer':       s => Buffer.from(s.slice(0, KEEP), 'utf8').toString('utf8'),
  'json':         s => JSON.parse(JSON.stringify(s.slice(0, KEEP))),
  'array-join':   s => Array.from(s.slice(0, KEEP)).join(''),
  'repeat1':      s => s.slice(0, KEEP).repeat(1),
  'padEnd':       s => s.slice(0, KEEP).padEnd(KEEP),
  'normalize':    s => s.slice(0, KEEP).normalize(),
};

I had put my money on the template literal and on String(). Both read like "construct a fresh string." Both sail straight through.

So do '' + s, repeat(1), and padEnd called with the string's existing length. From the engine's point of view, returning the input unchanged is the correct answer whenever the output would be identical. The copy we were counting on as a side effect is precisely what the optimizer exists to remove.

What works is only the rewrites that force the internal representation to be rebuilt: prepend a character and strip it, explode into an array and rejoin, round-trip through bytes or JSON. Every one of them looks like pointless work, and every one of them is the first thing a reviewer would delete.

The fastest fix is the one that looks most wasteful

I measured the cost too: 200 characters extracted from a 256 KB source, ten thousand times, median of seven runs.

The scrappiest-looking option, (' ' + x).slice(1), is the fastest of the working set — roughly 2.4 times quicker than the Buffer round-trip.

Meanwhile the two techniques that feel the most thorough, going through a character array, are more than ten times slower than that. Reasonable in hindsight, but I did not expect confidence and speed to sort in opposite directions.

The catch with (' ' + x).slice(1) is that nothing in the code explains why it exists. My compromise: Buffer round-trip wherever the cost is invisible, (' ' + x).slice(1) only on hot paths, and a comment either way.

Putting it in a tool wrapper and measuring again

Everything so far is synthetic. Here is the same question shaped like a real wrapper: a tool that returns 1 MB, called 200 times, with a 400-character preview kept in history.

// exp10.mjs — node --expose-gc exp10.mjs [naive|flatten]
const FLATTEN = process.argv[2] === 'flatten';
const CALLS = 200;
const OUT = 1024 * 1024;
const KEEP = 400;
 
function truncatePreview(s, keep) {
  if (s.length <= keep) return s;
  const head = s.slice(0, keep);
  // head still references the whole parent; the round-trip cuts it loose.
  return FLATTEN ? Buffer.from(head, 'utf8').toString('utf8') : head;
}
 
async function callTool(i) {
  return 'L'.repeat(OUT) + '\n#' + i;   // stand-in for a log-reading tool
}
 
const history = [];
global.gc();
const t0 = Date.now();
 
for (let i = 0; i < CALLS; i++) {
  const out = await callTool(i);
  history.push({ tool: 'read_log', preview: truncatePreview(out, KEEP), fullBytes: out.length });
}
 
global.gc();
const m = process.memoryUsage();
console.log(FLATTEN ? 'flatten' : 'naive  ',
            'heapUsed =', (m.heapUsed / 1048576).toFixed(1), 'MB',
            '/ rss =', (m.rss / 1048576).toFixed(1), 'MB',
            '/ elapsed =', Date.now() - t0, 'ms');

Four runs each, with almost no variance.

Identical information retained. About 44 times less heap, about 5.6 times less RSS.

The bill is 16 milliseconds across 200 calls — 0.08 ms each. Against the latency of the tool itself, that is noise.

Keeping fullBytes separately is deliberate. Throwing away the text while remembering how large it was lets you judge later whether the truncation was reasonable, and it makes it far less likely that the model reads a preview and assumes it is the whole thing.

Turning suspicion into evidence with a heap snapshot

"Memory is growing" is not the same as "sliced strings are the cause." Being able to count the retained parents turns the fix into something you can verify before and after.

Node ships v8.writeHeapSnapshot(), and the .heapsnapshot it produces is JSON. No external tooling required.

Snapshot nodes have a type of sliced string, and an internal edge from each one points at its parent. Follow that edge, filter to parents above a size floor, and sum.

#!/usr/bin/env node
// sliced_audit.mjs
// Count sliced strings in a heap snapshot and the parents they keep alive.
// Usage: node sliced_audit.mjs <snapshot> [min parent bytes] [budget MB]
import fs from 'node:fs';
 
const file = process.argv[2];
const PARENT_MIN_BYTES = Number(process.argv[3] || 262_144); // only parents >= 256 KB
const BUDGET_MB = Number(process.argv[4] || 8);
 
const snap = JSON.parse(fs.readFileSync(file, 'utf8'));
const { node_fields: nf, node_types: nt, edge_fields: ef, edge_types: et } = snap.snapshot.meta;
const NS = nf.length, ES = ef.length;
const TYPE = nf.indexOf('type'), NAME = nf.indexOf('name'),
      SIZE = nf.indexOf('self_size'), ECOUNT = nf.indexOf('edge_count');
const ETYPE = ef.indexOf('type'), ETO = ef.indexOf('to_node');
const nodeTypes = nt[0], edgeTypes = et[0];
 
// Precompute node index -> first edge index. Counting from the top each time is O(n^2).
const edgeStart = new Uint32Array(snap.nodes.length / NS);
for (let i = 0, o = 0, e = 0; o < snap.nodes.length; o += NS, i++) {
  edgeStart[i] = e;
  e += snap.nodes[o + ECOUNT];
}
 
// A Map keyed by parent offset prevents double counting when several slices share a parent.
const parents = new Map();
let sliced = 0;
 
for (let i = 0, o = 0; o < snap.nodes.length; o += NS, i++) {
  if (nodeTypes[snap.nodes[o + TYPE]] !== 'sliced string') continue;
  sliced++;
  const base = edgeStart[i] * ES;
  for (let k = 0; k < snap.nodes[o + ECOUNT]; k++) {
    const eo = base + k * ES;
    if (edgeTypes[snap.edges[eo + ETYPE]] !== 'internal') continue;
    const po = snap.edges[eo + ETO];
    const bytes = snap.nodes[po + SIZE];
    if (bytes >= PARENT_MIN_BYTES) parents.set(po, bytes);
  }
}
 
let retained = 0;
const samples = [];
for (const [po, bytes] of parents) {
  retained += bytes;
  if (samples.length < 3) {
    samples.push({
      parentMB: +(bytes / 1048576).toFixed(2),
      head: (snap.strings[snap.nodes[po + NAME]] || '').slice(0, 32),
    });
  }
}
 
const retainedMB = +(retained / 1048576).toFixed(2);
console.log(JSON.stringify({
  slicedStrings: sliced, retainedParents: parents.size, retainedMB, budgetMB: BUDGET_MB, samples,
}));
 
process.exit(retainedMB > BUDGET_MB ? 1 : 0);

The edgeStart table is the part worth keeping. Snapshot edges live in one flat array for the whole heap, so finding a given node's edges means summing every preceding node's edge count. Written naively that is a nested loop, and past a few tens of thousands of nodes the audit itself starts taking tens of seconds.

Run against a process that stacked fifty 2 MB outputs:

$ node sliced_audit.mjs naive.heapsnapshot
{"slicedStrings":54,"retainedParents":50,"retainedMB":100,"budgetMB":8,
 "samples":[{"parentMB":2,"head":"LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL"}, ...]}
exit=1

$ node sliced_audit.mjs flatten.heapsnapshot
{"slicedStrings":5,"retainedParents":1,"retainedMB":2,"budgetMB":8,"samples":[...]}
exit=0

Fifty parents, 100 MB, right there in the output. The flattened build shows one parent and 2 MB. The head field gives you the first characters of what is being retained, which is usually enough to identify the offending tool.

The audit took 0.09–0.10 seconds against a 5 MB snapshot. Cheap enough for CI.

One caveat: slicedStrings will never be zero. Libraries slice short strings constantly and that is entirely fine. The number that carries signal is the summed size of parents above your floor, not the count.

In Python the trap sits one step to the side

I ran the same shape in Python, and the answer flips. Slicing a str genuinely copies.

import tracemalloc
SIZE, N, KEEP = 4 * 1024 * 1024, 20, 200
tracemalloc.start()
base = tracemalloc.get_traced_memory()[0]
kept = [('x' * SIZE + '#' + str(i))[:KEEP] for i in range(N)]
cur = tracemalloc.get_traced_memory()[0]
print(f"retained = {(cur - base) / 1048576:.2f} MB")

Python 3.10.12 reports retained = 4.00 MB — one parent's worth, with no accumulation.

Swap in a memoryview over bytes and the trap reappears.

kept.append(memoryview(full)[:KEEP])   # full is bytes

That one reports retained = 80.01 MB. A memoryview is a zero-copy view by definition, so this is the documented behavior rather than a surprise — but it is the same failure mode wearing different clothes.

The general lesson is that this is not a Node defect. Every language draws the line between "slicing copies" and "slicing views" somewhere, and it is worth knowing where yours draws it before you store the result.

Anthropic shipped a fix for exactly this shape of bug on their side, where the untruncated text of a truncated MCP tool result stayed in memory for the life of the session. If the official implementation could hold on to it, there is no reason to assume your own wrapper does not.

What to do before you leave a process running

Three habits are all I keep now.

1. Funnel every truncation through one function. Scattering the flattening across individual slice calls guarantees you will miss one. Have a single truncateForHistory() and forbid pushing strings into history anywhere else.
2. Wire writeHeapSnapshot() into long-lived processes. Dumping a snapshot on a signal is enough. Being able to grab the artifact the moment things look wrong is what turns diagnosis from guesswork into measurement.
3. Run the audit with a budget in CI. The script above exits 1 when the total crosses your ceiling. Expressing the threshold in bytes rather than in counts turned out to be the practical choice.

And no, you should not flatten every slice. If the substring is consumed and discarded immediately, flattening is pure overhead. It only earns its keep when the result goes into something long-lived — history, caches, metric labels. Going back through the code I write as an indie developer, those three covered every real case.

If you have an MCP server or an agent sitting resident right now, start by driving a few hundred calls through it under node --expose-gc and watching heapUsed. If it climbs, the truncation line is the first place I would look.