Same Content, Different SHA-256: Making Pinned Plugin Archives Reproducible

Ordering, compression level, and extra fields all matter

Timestamps are not the only variable. Holding mtimes fixed and changing only the order in which files are fed to zip:

find ord -exec touch -d @1000000000 {} +
( cd ord && find . -type f | LC_ALL=C sort    | zip -qX ../ord_sorted.zip -@ )
( cd ord && find . -type f | LC_ALL=C sort -r | zip -qX ../ord_rev.zip    -@ )

Same size, different hash. zip -r walks the tree in whatever order the filesystem returns, which depends on directory internals — stable on one machine, not something to rely on across machines.

Compression settings behave the same way:

Three things fall out of that table.

1. -X (drop extra fields) took the archive from 21,891 to 21,579 bytes. The UT and ux (UID/GID) fields for six files accounted for 312 bytes. Removing owner IDs from a distributed artifact is desirable on its own merits.
2. The default compression level is -6, so plain -qX and -qX -6 hashed identically.
3. -6 and -9 produced the same 21,579 bytes but different hashes. The deflate search strategy changed, yielding a different byte stream at the same length. Size equality is not a proxy for identity — something I would not have caught without measuring.

Twenty builds, one hash

The fix reduces to four operations: normalize mtimes to a fixed value, fix entry order in a locale-independent way, strip extra fields, and pin the compression level.

#!/usr/bin/env bash
# pack.sh — build a deterministic plugin zip. Same contents, same SHA-256, always.
set -euo pipefail
SRC="${1:?usage: pack.sh <plugin-dir> <out.zip>}"
OUT="${2:?}"
 
[ -f "$SRC/plugin.json" ] || { echo "no plugin.json in: $SRC" >&2; exit 1; }
 
STAGE="$(mktemp -d)"
trap 'rm -rf "$STAGE"' EXIT
cp -a "$SRC/." "$STAGE/"
 
# Drop things that must never ship (.git alone guarantees a changing hash)
rm -rf "$STAGE/.git" "$STAGE/node_modules"
find "$STAGE" -name '.DS_Store' -delete
 
# 1) Normalize mtimes; -h covers symlinks themselves
find "$STAGE" -exec touch -h -d @1000000000 {} +
 
# 2) Fix entry order in byte order, immune to locale collation
# 3) -X strips UID/GID and the extended timestamp
# 4) Pin the compression level explicitly, so a future default change cannot move the hash
( cd "$STAGE" && find . -type f -print | LC_ALL=C sort | zip -qX -9 "$OLDPWD/$OUT" -@ )
 
printf '%s  %s\n' "$(sha256sum "$OUT" | cut -d' ' -f1)" "$OUT"

I verified it by scrambling mtimes before every iteration, specifically to defeat the accidental-match case:

for i in $(seq 20); do
  find det -type f -exec touch -d "@$((1700000000 + RANDOM))" {} +
  ./pack.sh det det.zip
done

All twenty runs produced fc6fbadb81cc38dd... at 21,579 bytes. Twenty out of twenty.

The separate-checkout case passes too. Forcing one tree to 2025-era mtimes and the other to 2026-era mtimes still yields:

fc6fbadb81cc38dd1ba7d51347c1755ffcf9fc8bd8dc85cb0970f038ee954355  a.zip
fc6fbadb81cc38dd1ba7d51347c1755ffcf9fc8bd8dc85cb0970f038ee954355  b.zip

A single changed byte still moves the hash, which is the whole point. Appending one character to summarize.md took fc6fbadb81cc38dd to 1be5cbf22e3fea20.

What the pin does not cover

Once the producing side is deterministic, tighten the consuming side.

#!/usr/bin/env bash
# verify.sh — fixed source, contents checked before use
set -euo pipefail
URL="${1:?usage: verify.sh <url> <expected-sha256>}"
WANT="${2:?}"
 
TMP="$(mktemp -d)"
trap 'rm -rf "$TMP"' EXIT
 
curl -fsSL --proto '=https' --tlsv1.2 -o "$TMP/p.zip" "$URL"
 
GOT="$(sha256sum "$TMP/p.zip" | cut -d' ' -f1)"
if [ "$GOT" != "$WANT" ]; then
  echo "SHA-256 mismatch" >&2
  echo "  expected: $WANT" >&2
  echo "  actual  : $GOT"  >&2
  exit 1
fi
echo "SHA-256 verified: $GOT"

--proto '=https' closes the redirect-to-plaintext path. The hash would catch tampering either way, but detecting a problem and refusing to create the opportunity are different things.

Beyond that, it helps to know the pin's boundaries. The clearest measurable case is packager differences. Given identical input, identical timestamps, and identical ordering, Python's zipfile and Info-ZIP disagree:

Identical contents, different hash, different size. A mismatch is therefore not evidence of tampering — someone may simply have swapped the packaging tool. If you rely on pinning, the packaging procedure itself belongs in version control alongside the artifact. That is a large part of why pack.sh ships with the plugin.

The same reasoning extends outward. If your plugin shells out to npx at runtime, that dependency resolves independently of the archive hash. Archive identity is not runtime identity. For the environmental assumptions plugins tend to carry silently, see four implicit assumptions in plugin portability.

Measuring an archive before you pin it

Everything so far has been about the side that builds. In practice you pin other people's archives more often than your own, and what you want to know then isn't what's inside — it's whether this distribution was produced in a form that can be rebuilt byte-for-byte for the next release.

For a while I settled for reading unzip -l. That listing gives names, sizes, and timestamps. The things that actually move the hash — extra fields and entry order — never appear in it. I was making a judgment about bytes I had never looked at.

So I wrote a small check that reads only the central directory and counts the things that will move the hash on a repack. It never extracts.

#!/usr/bin/env python3
"""Audit a zip for pinning: look only at what threatens a byte-identical rebuild."""
import sys, zipfile, struct
 
KNOWN = {0x5455: "UT (extended timestamp)", 0x7875: "ux (UID/GID)",
         0x000a: "NTFS times", 0x0001: "Zip64", 0x9901: "AES"}
 
def parse_extra(blob):
    """Split the extra field into (id, body) pairs; stop at the first malformed record."""
    out, i = [], 0
    while i + 4 <= len(blob):
        hid, size = struct.unpack_from("<HH", blob, i)
        body = blob[i + 4: i + 4 + size]
        if len(body) != size:
            break
        out.append((hid, body))
        i += 4 + size
    return out
 
def uid_gid(body):
    """Read UID/GID out of ux (0x7875). Anything but version 1 is left alone."""
    if not body or body[0] != 1:
        return None
    i, vals = 1, []
    for _ in range(2):
        if i >= len(body):
            return None
        n = body[i]; i += 1
        if i + n > len(body):
            return None
        vals.append(int.from_bytes(body[i:i + n], "little")); i += n
    return tuple(vals)
 
def inspect(path):
    with zipfile.ZipFile(path) as z:
        infos = z.infolist()
    names = [i.filename for i in infos]
    stamps = {i.date_time for i in infos}
    extra_bytes = sum(len(i.extra) for i in infos)
    ids, owners = {}, set()
    for i in infos:
        for hid, body in parse_extra(i.extra):
            ids[hid] = ids.get(hid, 0) + 1
            if hid == 0x7875:
                og = uid_gid(body)
                if og:
                    owners.add(og)
    ordered = names == sorted(names)
    print(f"# {path}")
    print(f"  entries          : {len(infos)} ({sum(1 for n in names if n.endswith('/'))} directories)")
    print(f"  distinct DOS ts  : {len(stamps)}")
    print(f"  extra field bytes: {extra_bytes}")
    for hid, cnt in sorted(ids.items()):
        print(f"    - 0x{hid:04x} {KNOWN.get(hid, 'unknown'):<24} {cnt} entries")
    print(f"  UID/GID          : {sorted(owners) if owners else 'none'}")
    print(f"  compression      : {sorted({i.compress_type for i in infos})}")
    print(f"  byte-sorted order: {ordered}")
    risky = (len(stamps) > 1) or (0x5455 in ids) or (0x7875 in ids) or (not ordered)
    print(f"  verdict          : {'WARN - a repack will move the hash' if risky else 'OK - normalized'}")
    return 1 if risky else 0
 
if __name__ == "__main__":
    sys.exit(max(inspect(p) for p in sys.argv[1:]))

I built a small plugin tree (6 files, 5 directories), packed it once with a plain zip -qr and once through pack.sh, and ran the check on both. This is the actual output:

# naive.zip
  entries          : 11 (5 directories)
  distinct DOS ts  : 1
  extra field bytes: 264
    - 0x5455 UT (extended timestamp)   11 entries
    - 0x7875 ux (UID/GID)              11 entries
  UID/GID          : [(1199, 1199)]
  compression      : [0, 8]
  byte-sorted order: False
  verdict          : WARN - a repack will move the hash
# norm.zip
  entries          : 6 (0 directories)
  distinct DOS ts  : 1
  extra field bytes: 0
  UID/GID          : none
  compression      : [8]
  byte-sorted order: True
  verdict          : OK - normalized

The line worth staring at is that the naive archive also reports a single distinct DOS timestamp. Files written in one sitting land inside the same two-second window, so counting timestamps tells you nothing about the risk. The accidental-match trap from earlier in this article reappears here, on the auditing side. That's why the verdict keys off UT, ux, and entry order instead of the timestamps you can see.

Finding my own (1199, 1199) sitting in ux was its own small surprise. Beyond reproducibility, I'd rather not ship account numbers from my machine, and -X handles that too.

Then I isolated which of the four normalization steps is actually load-bearing. Two trees whose mtimes sit about four months apart — standing in for two independent checkouts — packed under three different conditions:

The middle row is the one I'd underlined. Stripping UT with -X still leaves the DOS timestamps in the local file headers and the central directory, so the hashes stay apart. Dropping extra fields and fixing times are two different jobs. Ordering and compression level narrow the variance; the determinism itself rests on that single touch line.

You cannot pin what you never measured. Wiring this check into CI against the output of pack.sh, and letting the exit code fail the build, means that dropping one of the four steps some months from now breaks on my machine rather than in someone else's install.

Putting it into practice

For an internal distribution, five things are worth setting up before you publish anything.

1. Commit pack.sh and call it from CI. Never hand-build the zip. Fixing the packager settles half the reproducibility question by itself.
2. Ship the hash as a manifest. Put plugin.zip.sha256 next to plugin.zip and move the URL and the hash in the same commit, so neither can be replaced alone.
3. Put the version in the URL. Pinning a mutable entry point like latest to a SHA-256 breaks every consumer on the next release. Pair a fixed URL with a fixed hash and publish a new pair to migrate.
4. Ignore the verification cost. SHA-256 over a 50 MB file took 224 ms on this VM. Plugin archives are typically two orders of magnitude smaller, so verifying on every install is imperceptible.
5. Always pass zip -X. Beyond reproducibility, it keeps UID and GID out of the artifact.

One thing I could not verify: locale-dependent collation. I built a tree containing a Japanese-named directory to test it, but this VM only carried C/POSIX locales (locale -a returned four entries), so I could not reproduce a difference. What I did measure is that entry order changes the hash, and there is no good reason to let the build environment's default collation decide it — hence LC_ALL=C.

Zip distribution is new enough that the tooling may eventually normalize archives on its own. Even then, a deterministic producer costs nothing.

Where to go next

If you are evaluating zip distribution, write pack.sh before you fill in a single pin. Discovering the reproducibility problem after publishing means your users are the ones who hit it. Reversing the order removes the failure entirely.

I had let the word "hash" do too much work in my head — same input, same output, obviously. The definition of "input" quietly included wall-clock time and filesystem ordering. That is the kind of assumption you only see once you measure it.

On choosing where plugins come from in the first place, finding and evaluating plugin marketplaces covers the upstream half of this problem. For current specifics, check the Claude Code documentation.