The Morning I Hit Cloudflare Workers' 62 MiB Limit, and the Content Split Architecture I Rebuilt for 5,000 Articles

Choosing to split metadata and body

The pattern goes by "Content Split Architecture" in CMS circles, and translating it to Cloudflare Workers + OpenNext narrowed the design choices down to three.

1. Keep articles.json to metadata only: title, slug, category, level, premium, date, description, tags, highlights. At roughly 800 bytes per article, 4,500 entries fit in about 3.6 MB.
2. Write HTML bodies to public/content/articles/{locale}/{category}/{slug}.html as individual files. Files under public/ are served by Cloudflare's static assets layer and never enter the Worker bundle.
3. At runtime, fetch bodies via the ASSETS binding (getCloudflareContext().env.ASSETS.fetch()) and stream the HTML into the Server Component with dangerouslySetInnerHTML.

After the switch, the Worker bundle holds only ~3.6 MB of metadata plus Next.js code; the 14 MB worth of HTML escapes to per-asset static delivery. As a bonus, individual asset delivery means rendering one article does not push any of the other 4,499 articles' bodies across the network.

Wiring the two-stage output into generate-content.mjs

The most useful change in code is rewriting generate-content.mjs so each MDX produces two outputs: an entry in the metadata index and an HTML file on disk.

// New generate-content.mjs (abbreviated)
import { promises as fs } from "node:fs";
import path from "node:path";
import matter from "gray-matter";
import { mdxToHtml } from "./lib/mdx.mjs";
 
const metaIndex = [];
for (const locale of ["ja", "en"]) {
  const root = `content/articles/${locale}`;
  for (const file of await walk(root)) {
    const raw = await fs.readFile(file, "utf8");
    const { data, content } = matter(raw);
    const slug = path.basename(file, ".mdx");
    const category = path.basename(path.dirname(file));
    const html = await mdxToHtml(content);
 
    // (1) metadata-only entry into the index
    metaIndex.push({ ...data, locale, category, slug });
 
    // (2) write body HTML as a per-article file under public/
    const outDir = `public/content/articles/${locale}/${category}`;
    await fs.mkdir(outDir, { recursive: true });
    await fs.writeFile(`${outDir}/${slug}.html`, html, "utf8");
  }
}
 
await fs.writeFile(
  "src/generated/articles.json",
  JSON.stringify(metaIndex)
);

The moment this ships, articles.json shrinks from 18 MB to 3.6 MB. The per-article HTML files average around 4 KB each, totaling about 14 MB now living outside the Worker bundle.

The reason I write to public/ is that Cloudflare Pages / Workers Assets automatically serves whatever lands there as static assets. With OpenNext, configure the [assets] section of wrangler.toml, and the Worker can reach the files via env.ASSETS.fetch().

Fetching the body through the ASSETS binding

I added getArticleContent() in src/lib/content.ts to consolidate the call site. One thing I strongly recommend: do not let the Worker call fetch() against your own hostname for the static asset. That tends to loop. Use env.ASSETS.fetch() instead.

// src/lib/content.ts
import { getCloudflareContext } from "@opennextjs/cloudflare";
 
export async function getArticleContent(
  locale: string,
  category: string,
  slug: string
): Promise<string | null> {
  const { env } = getCloudflareContext();
  const url = `https://placeholder/content/articles/${locale}/${category}/${slug}.html`;
  const res = await env.ASSETS.fetch(url);
  if (!res.ok) return null;
  return await res.text();
}

The https://placeholder host is essentially ignored by the ASSETS binding (it does not look at the hostname), but writing something readable there makes the file easier to follow six months later. The Server Component side just calls it like this.

const html = await getArticleContent(locale, category, slug);
if (!html) notFound();
return <article dangerouslySetInnerHTML={{ __html: html }} />;

Whether getCloudflareContext() returns synchronously or as a Promise depends on the @opennextjs/cloudflare version, so I recommend pinning the version and verifying against the release notes. On 1.x in my environment, it returned { env, ctx } synchronously.

Letting Cloudflare CI run it automatically via prebuild

Locally npm run build runs everything in one shot, but Cloudflare's CI also needs to run generate-content.mjs before the Next.js build kicks in. I have found that hooking it into the prebuild script in package.json is the lowest-friction way.

{
  "scripts": {
    "prebuild": "node generate-content.mjs",
    "build": "next build && open-next build"
  }
}

That single hook causes the Cloudflare CI to run prebuild before every npm run build, regenerating both src/generated/articles.json and public/content/articles/.../*.html. The recommendation here is to add both src/generated/ and public/content/ to .gitignore and treat them as build-time outputs. Committing thousands of tiny HTML files into git makes git pull noticeably slower as the delta size compounds.

What the first month of numbers actually looked like

I rolled this out site-by-site, starting with Claude Lab and giving each of the other three sites a one-week stagger. After one month of running with the new architecture, the observed numbers were:

• Worker bundle: 18 MB → 2.4 MB pre-Brotli, 4.8 MB → 0.9 MB post-Brotli. The headroom against the 62 MiB cap went from "uncomfortably close" to roughly 60x.
• Local npm run build time: 78 s → 62 s. Writing thousands of HTML files added some cost, but cutting the giant JSON serialization more than paid it back, netting around 20% faster.
• Cloudflare deployment success rate over the past 30 days: 100%. Before the switch I was hitting the 62 MiB cap once or twice per month and needed to retry.
• Article-page TTFB at the Tokyo edge with a cache hit: 38 ms → 41 ms. The one extra hop into ASSETS adds a tiny amount, but in practice the difference is imperceptible.

The public/content/ tree grew to about 9,000 HTML files (4,500 articles × 2 locales), but Cloudflare's object storage scales well past that, so I do not see this as a near-term concern.

Self-fetch and getCloudflareContext: two boundaries that bite

Three concrete pitfalls I tripped over while implementing this. Writing them down here so future-me, when touching the same code, has the line drawn back in.

The first pitfall is calling fetch("https://my-site.example/content/...") from inside the Worker. That triggers a recursive subrequest into the same Worker, hitting either the subrequest cap or the CPU time limit. Use env.ASSETS.fetch().

The second pitfall is calling getCloudflareContext() outside of a request boundary. If the article page is export const dynamic = "force-static", the initial build runs without a Cloudflare context and the function throws Cannot access Cloudflare context outside of a request. I now lean toward dynamic = "force-dynamic" or revalidate = 60 (ISR) for article pages.

The third pitfall is committing public/content/ into git. I made this mistake on my first pass, added articles.json to .gitignore but forgot the HTML tree, and the repo size shot up so much that git clone started taking visibly longer. The fix: both directories belong in .gitignore.

Rolling the same design out to the other three sites

For an indie operator running four blogs in parallel, copy-pasting is the realistic answer, but per-site differences need to be respected. The order I used was:

1. Run Claude Lab on the new architecture for two weeks, long enough to hit and resolve all three pitfalls above.
2. Bring generate-content.mjs and src/lib/content.ts over to Gemini Lab. Adjust only the category names and OG image paths, then verify a small batch end-to-end.
3. After one week of Gemini Lab running smoothly, apply the same change to Antigravity Lab and Rork Lab on the same day.
4. Confirm public/content/ and src/generated/ are both in .gitignore on every site.

The generate-content.mjs ended up nearly identical across the four sites, but mdxToHtml differs because each site's remark plugin chain is slightly different. The SKILL.md "count-match check" (find content/articles/ja -name "*.mdx" | wc -l equals the equivalent for en) still matters after the Content Split rewrite, because a missing English MDX surfaces as a hard 404 on locale switch. That check stays in the workflow.

Looking ahead from 5,000 to 10,000 articles

Even metadata-only, the JSON is roughly 4 MB at 5,000 articles. It will grow linearly: 8 MB at 10,000, 16 MB at 20,000. Somewhere around 22,000 articles I will be back on uncomfortably-close terms with the 62 MiB cap, so I need to plan the next step before then.

The two options I currently consider strongest:

1. Split the metadata JSON further by category or by month, and dynamically load only the slice needed. The list page for /articles/claude-code would only need articles-by-category/claude-code.json. Even related-article extraction can usually live within a single category file.
2. Move metadata to D1 (Cloudflare's SQLite) and remove it from the Worker bundle entirely. Search-experience improvements come along for the ride, but the build pipeline gains a meaningful amount of complexity.

Reaching 10,000 articles is roughly six months to a year out, so I plan to start with option 1 and reach for option 2 only if it proves insufficient.

Every design choice above is, in the end, a guide line for whoever touches this code next, including future-me. The temple-carpenter grandfathers I started this article with built things that still stand straight because the lines they drew were for someone else, often someone they would never meet. I hope the architecture I rebuilt against the 62 MiB ceiling will be that kind of line for me, six months from now. Thanks for reading.