Yarn berry vendored and hosted pins of native-addon packages (nan, bufferutil, utf-8-validate, node-addon-api) keep the registry entry's implicit node-gyp: "npm:latest" dependency, so vendored installs and hardened hosted installs fail YN0028 #737

[agent] Found by the scheduled Yarn Berry (2+) bug-hunt routine (ledger #305).

Summary

When yarn resolves a package through the npm registry and the registry metadata marks it as a gyp package, yarn adds an implicit node-gyp: "npm:latest" dependency to the lock entry. When yarn resolves the same package through a tarball URL or a file: locator, it reads the manifest from the tarball and adds no such dependency.

Both Berry pin writers build the pinned entry by copying the registry entry's body, so the implicit dependencies: node-gyp: "npm:latest" is copied too:

• hosted: crates/socket-patch-core/src/patch/redirect/mod.rs:3788 ("dependencies, bin, languageName — carries over verbatim")
• vendored: crates/socket-patch-core/src/vendor/yarn_berry_lock.rs:336 and carried_sections at :1303

On the next resolution, yarn drops that dependency together with the whole node-gyp subtree (node-gyp, tar, undici, which, nopt and others; about 20 entries), so the lock no longer matches.

This has the same root cause as #718 (copying the registry body instead of rendering the entry yarn writes for the new locator), but it affects a different field. #719 doesn't fix it: I built #719's head 635ca29 and it still fails (see the table below). Once #719 lands and closes #718, this case stays broken, so I'm filing it separately.

Impact

The affected packages are very common: every package whose published tarball ships a binding.gyp with no install script. A survey with yarn 4.12.0 (comparing the npm: entry with the tarball-URL entry) found the extra dependency on nan, node-addon-api, bufferutil, utf-8-validate (both are optional deps of ws), microtime, iconv and ref-napi. bindings and deasync are not affected.

• Vendored: every fresh-checkout yarn install --immutable fails YN0028, and so does every CI run.
• Hosted: a plain --immutable install passes, but hardened mode (enableHardenedMode, which yarn turns on automatically for public PRs in GitHub Actions) or --refresh-lockfile fails YN0028.

Repro (yarn 4.12.0, node-modules linker)

mkdir app && cd app
echo '{"name":"app","private":true,"dependencies":{"nan":"2.22.0"}}' > package.json
printf 'nodeLinker: node-modules\nenableGlobalCache: false\n' > .yarnrc.yml
yarn install && git init -q && git add -A && git commit -qm init
socket-patch scan --mode vendored --yes --api-url <mock> --org org --api-token x   # or --mode hosted
git add -A && git commit -qm patched
git clone -q . ../fresh && cd ../fresh
yarn install --immutable                              # vendored: YN0028
YARN_ENABLE_HARDENED_MODE=1 yarn install --immutable  # hosted and vendored: YN0028

The patch API is a local mock that serves a patched tarball with its real yarnBerry10c0 checksum, which is bootstrapped with a real yarn resolutions: file: install. That's the same harness as #718.

The pinned entry socket-patch writes (hosted):

"nan@http://127.0.0.1:18555/patch/npm/nan/2.22.0/tok/<uuid>/nan-2.22.0.tgz":
  version: 2.22.0
  resolution: "nan@http://…/nan-2.22.0.tgz"
  dependencies:
    node-gyp: "npm:latest"        # <- yarn removes this, plus the whole node-gyp subtree
  checksum: 10c0/e60f0a30…

A hardened install (with immutable off) rewrites it without the dependencies: block and deletes node-gyp@npm:latest, tar@npm:^7.5.7, undici@npm:^8.4.1 and the rest of that subtree.

Expected vs actual

• Expected: docs/ecosystems.md (yarn-berry hosted) says "hardened mode accepts it", and the vendored flow promises a lock that a fresh yarn install --immutable accepts (CLI_CONTRACT: the rewritten lock must install under frozen/immutable installs).
• Actual: YN0028 on the cells below.

Cells (Linux, Node 22, main 045d7ec)

Each failing cell reproduced at least twice. Hosted rollback of the nan pin restores both files byte-exactly, so it isn't affected.

First bad: hosted, #465 (203e092), because release 4.0.0's npm: locator pin kept yarn's registry resolution. I couldn't drive release 4.0.0's vendored flow against the mock.

Suggested direction: derive dependencies / peerDependencies / dependenciesMeta (not only bin) from the served tarball's package.json, the way yarn's tarball/file fetchers do, rather than carrying the registry entry's maps over. Probe branches for macOS and Windows weren't possible this run (branch deletes are blocked in the routine's sandbox).