build: use Yarn JsZipImpl for node-modules link to fix arm32 OOM (#51231)

* build: use Yarn JsZipImpl for node-modules link step

Patch the vendored .yarn/releases/yarn-4.12.0.cjs so the node-modules
(and pnpm-loose) linker constructs its read-only ZipOpenFS with
customZipImplementation = JsZipImpl instead of the default WASM
LibZipImpl.

LibZipImpl loads each cache zip fully into the Emscripten WASM heap and
malloc's a WASM buffer per file read. With up to 80 zips held open, the
32-bit arm32v7 test container intermittently fails to allocate the ~9 MB
buffer for typescript/lib/typescript.js. Yarn's cross-FS copyFilePromise
swallows the real error and surfaces it as a generic
"EINVAL: invalid argument, copyfile", which has been failing ~1-in-3
linux-arm test shards at Install Dependencies since 2026-04-13.

JsZipImpl opens zips by fd, reads only the central directory, and pulls
individual entries into plain Node Buffers — no WASM heap involved.
There is no .yarnrc.yml or env knob for this, so the vendored release is
edited directly. .claude/README.md documents the patch and how to
re-apply it on Yarn upgrades.

Refs: yarnpkg/berry#3972, yarnpkg/berry#6722, yarnpkg/berry#6550

Co-authored-by: Sam Attard <sattard@anthropic.com>

* docs: move JsZipImpl patch notes to .yarn/README.md

Relocate the patch rationale next to the vendored release it documents,
reword the intro for its new home, and update the header comment in
yarn-4.12.0.cjs to point at .yarn/README.md.

Co-authored-by: Sam Attard <sattard@anthropic.com>

---------

Co-authored-by: trop[bot] <37223003+trop[bot]@users.noreply.github.com>
Co-authored-by: Sam Attard <sattard@anthropic.com>
This commit is contained in:
trop[bot]
2026-04-22 12:27:05 +02:00
committed by GitHub
parent 9d564abf41
commit f111adcfc0
2 changed files with 87 additions and 2 deletions

82
.yarn/README.md Normal file
View File

@@ -0,0 +1,82 @@
# Vendored Yarn release
This directory holds the Yarn release used by this repo (`yarnPath` in
`.yarnrc.yml`). The release file is checked in so every contributor and CI job
runs the exact same Yarn, and so we can carry small local patches when needed.
`releases/yarn-4.12.0.cjs` currently carries one such patch, described below.
If you bump the Yarn version, read the **Upgrading Yarn** section first.
## Patch: use `JsZipImpl` for the node-modules link step
### What changed
Two call sites in `releases/yarn-4.12.0.cjs` are modified so the
`node-modules` linker (and the `pnpm`-loose linker) construct their read-only
`ZipOpenFS` with `customZipImplementation: ST` — Yarn's pure-JS `JsZipImpl`
instead of falling through to the default WASM-backed `LibZipImpl`:
```text
new $f({maxOpenFiles:80,readOnlyArchives:!0})
→ new $f({maxOpenFiles:80,readOnlyArchives:!0,customZipImplementation:ST})
```
A comment block at the top of the `.cjs` file marks the file as patched and
points back here.
### Why
On the `linux-arm` CI test shards we run a 32-bit `arm32v7` container. During
`yarn install`'s **Link step**, Yarn opens up to 80 cache zips concurrently.
With `LibZipImpl`, each open zip is `readFileSync`'d into a Node `Buffer`
**and copied again into the WASM linear memory**, and every file read does a
WASM `_malloc(size)` for the entry. The WASM heap has to grow as a single
contiguous region of the 32-bit address space; once enough zips are resident,
the `_malloc` for a large entry — most often `typescript/lib/typescript.js`
(~9 MB inside a ~22 MB zip) — fails.
Yarn's cross-FS `copyFilePromise` swallows the underlying error and re-throws
a generic one, so CI shows:
```text
YN0001: While persisting .../typescript-patch-...zip/node_modules/typescript/
EINVAL: invalid argument, copyfile '/node_modules/typescript/lib/typescript.js' -> '...'
```
The unmasked form (occasionally seen on `pdfjs-dist`) is the WASM-heap failure
string `Couldn't allocate enough memory`. This started failing ~1-in-3
`linux-arm / test` shards at **Install Dependencies** on 2026-04-13, after
[#50692](https://github.com/electron/electron/pull/50692) grew the cache enough
to push the 32-bit process over the edge nondeterministically — e.g.
[run 24739817558](https://github.com/electron/electron/actions/runs/24739817558/job/72380803746).
`JsZipImpl` avoids the problem entirely: it opens the zip by file descriptor,
reads only the central directory into memory, and `readSync`s individual
entries into ordinary Node `Buffer`s — **no WASM heap involved**. It is
read-only and path-based, which is exactly how the linker uses these archives.
There is no `.yarnrc.yml` setting or environment variable to select the zip
implementation (verified against the bundle), so editing the vendored release
is the only way to switch it short of re-implementing the linker in a plugin.
Upstream references:
[yarnpkg/berry#3972](https://github.com/yarnpkg/berry/issues/3972),
[yarnpkg/berry#6722](https://github.com/yarnpkg/berry/issues/6722),
[yarnpkg/berry#6550](https://github.com/yarnpkg/berry/issues/6550).
### Upgrading Yarn
When bumping `releases/yarn-*.cjs`:
1. Check whether upstream now defaults `readOnlyArchives` opens to `JsZipImpl`,
or exposes a config knob for the zip implementation. If so, drop this patch.
2. Otherwise, re-apply: search the new bundle for
`maxOpenFiles:80,readOnlyArchives:!0` (the surrounding minified identifiers
will differ) and add `,customZipImplementation:<JsZipImpl symbol>` — that
symbol is whatever the new bundle exports as `JsZipImpl` from
`@yarnpkg/libzip`.
3. Re-add the header comment pointing back to this README.
4. Verify with
`rm -rf node_modules spec/node_modules && node script/yarn.js install --immutable --mode=skip-build`
and confirm `node_modules/typescript/lib/typescript.js` is byte-identical to
an unpatched install.

File diff suppressed because one or more lines are too long