ci: centralize build-image SHA and pre-seed node-gyp headers (#51148)
* ci: centralize build-image SHA and pre-seed node-gyp headers
- Add .github/actions/build-image-sha as the single source of truth for
the ghcr.io/electron/build (and arch-tagged electron/test) image SHA,
with an optional override input for workflow_dispatch.
- Refactor build.yml, apply-patches.yml, build-git-cache.yml,
clean-src-cache.yml, clean-orphaned-cache-uploads.yml, and the three
publish workflows to resolve the SHA via a small ubuntu-slim setup job
instead of hardcoding it in each file.
- Bump the image to daad061f (electron/build-images#68, which pre-warms
the node-gyp header cache in the Linux images).
- Run the build.yml setup job on ubuntu-slim instead of ubuntu-latest.
- In install-dependencies (and the inline yarn installs in
pipeline-electron-lint and generate-types), link deps with
--mode=skip-build first, run `node-gyp install` with up to 3 retries
(5s backoff) to populate the header cache, then run the build phase.
This avoids the parallel-download race that intermittently fails the
first native-addon configure with an empty common.gypi on cold
macOS/Windows runners.
* ci: skip node-gyp header pre-seed on Linux
* ci: invoke node-gyp via its JS entrypoint for Windows compat
(cherry picked from commit f7ba34064e)
Clone angle from github.com/google/angle in fix-sync action
Co-authored-by: trop[bot] <37223003+trop[bot]@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
ci: build a patched siso for Windows builds (#51077)
* ci: build a patched siso for Windows builds
The Windows Chromium builds intermittently fail during manifest load
with 'The parameter is incorrect.' (ERROR_INVALID_PARAMETER) out of
bindflt.sys. Root cause is a handle-relative NtCreateFile race in
siso/toolsupport/ninjautil/file_parser.go, which opens each subninja
twice — once in the outer goroutine and once more per chunk for
ReadAt. (*os.File).ReadAt is documented as safe for concurrent use,
so the extra open is redundant and removing it both halves the
CreateFileW calls per subninja and sidesteps the race.
Add a new build-siso-windows job on ubuntu-latest (runs in parallel
with checkout-windows) that:
- reads chromium_version from DEPS and pulls the matching siso_version
SHA from the Chromium mirror's DEPS at that ref
- shallow-clones chromium.googlesource.com/build at that SHA
- applies the in-tree patches under .github/siso-patches/ via git am
- cross-compiles siso.exe for windows/amd64
- caches the binary keyed on siso SHA + sha256 of the patches, so
subsequent runs hit the cache and skip the clone/patch/build steps
- uploads the result as a siso-windows-amd64 artifact
The Windows build jobs now depend on build-siso-windows, download the
artifact into $RUNNER_TEMP/siso, and export SISO_PATH, which
depot_tools/siso.py already honors. Mirrored into windows-publish.yml
and the regenerated pipeline-segment-electron-publish.yml so release
builds pick it up too.
Notes: none
* ci: extract siso build into a reusable workflow segment
Move the build-siso-windows job body into
pipeline-segment-build-siso-windows.yml and call it from both build.yml
and windows-publish.yml via workflow_call. Also pin actions/cache to
v5.0.5 and add version comments next to the action SHAs introduced by
this change.
(cherry picked from commit 5523130c92)
* ci: use hermetic mac SDK for the release ffmpeg build
gn gen out/ffmpeg runs as a raw gn invocation, so it never receives the
mac_sdk_path arg that e build injects for out/Default. On macOS runners
that means out/Default builds against the hermetic build-tools SDK while
out/ffmpeg falls through to the runner's system Xcode SDK. Reuse the
value e build already wrote so both builds share the same sysroot.
Co-authored-by: Samuel Attard <sattard@anthropic.com>
* ci: copy hermetic SDK symlink into out/ffmpeg and rewrite path
mac_sdk_path must live under root_build_dir, so pointing out/ffmpeg at
//out/Default/... doesn't work. Copy the xcode_links symlink tree into
out/ffmpeg and rewrite the path. Gate on Darwin so Windows/Linux don't
run the sed/cp at all.
Co-authored-by: Samuel Attard <sattard@anthropic.com>
---------
Co-authored-by: trop[bot] <37223003+trop[bot]@users.noreply.github.com>
Co-authored-by: Samuel Attard <sattard@anthropic.com>
ci: make src-cache upload atomic and sweep orphaned temp files
The checkout action's cp of the ~6GB zstd archive directly to the final
path on the cache share is non-atomic; an interrupted copy or a
concurrent reader produces zstd "Read error (39): premature end" on
restore, and the truncated file then satisfies the existence check so
no later run repairs it.
Upload to a run-unique *.tar.upload-<run_id>-<attempt> temp name on the
share and mv to the final path, discarding our temp if a concurrent run
got there first. A new clean-orphaned-cache-uploads workflow removes
temp files older than 4h every 4 hours.
Co-authored-by: trop[bot] <37223003+trop[bot]@users.noreply.github.com>
Co-authored-by: Sam Attard <sattard@anthropic.com>
chore: harden GitHub Actions against script injection patterns (#50512)
* fix: harden GitHub Actions against script injection vulnerabilities
Replace direct ${{ }} expression interpolation in run: blocks with
environment variables to prevent script injection attacks. Changes:
- archaeologist-dig.yml: move clone_url, head.sha, base.ref to env vars
- non-maintainer-dependency-change.yml: move user.login to env var
- issue-unlabeled.yml: move toJSON(labels) to env var
- issue-labeled.yml: move issue.number to env var
- pipeline-electron-lint.yml: validate chromium_revision format
- cipd-install/action.yml: move all inputs to env vars and quote them
- set-chromium-cookie/action.yml: reference secrets via $ENV_VAR
- Add security comments to all 5 pull_request_target workflows
https://claude.ai/code/session_01UUWmLxn5hyyxrhK8rGxU2s
* fix: allow version strings in chromium_revision validation
The previous regex `^[a-f0-9]+$` only matched git SHAs but
chromium_revision is a version string like `148.0.7741.0`.
Broaden to `^[a-zA-Z0-9._-]+$` which still blocks shell
metacharacters.
https://claude.ai/code/session_01UUWmLxn5hyyxrhK8rGxU2s
---------
Co-authored-by: Claude <noreply@anthropic.com>
build: replace npx with lockfile-pinned binaries (#50598)
* build: replace npx with lockfile-pinned binaries
- nan-spec-runner: reorder yarn install first, invoke nan node-gyp bin directly
- publish-to-npm: use host npm with E404 try/catch (closes existing TODO)
- upload-symbols: add @sentry/cli devDep, invoke from node_modules/.bin
- remove script/lib/npx.py (dead since #48243)
* build: bump @sentry/cli to 1.70.0 for arm support
* build: bump @sentry/cli to 1.72.0, skip CDN download on test jobs
@sentry/cli fetches its platform binary from Sentry CDN at postinstall.
Only upload-symbols.py (release pipeline) needs the binary; set
SENTRYCLI_SKIP_DOWNLOAD=1 in the two test-segment workflows that
call install-dependencies. The 64k variant uses pre-built artifacts
and does not install deps.
ci: zstd-compress the src cache and drop the doubled win_toolchain (#50702)
* ci: shrink src cache and fix Windows tar cleanup
- Exclude platform-specific toolchains (llvm-build, rust-toolchain) from
the src cache; all platforms now fetch them via fix-sync post-restore
- Exclude unused test data and benchmarks: blink/web_tests, jetstream,
speedometer, catapult/tracing/test_data, swiftshader/tests/regres
- Fix Windows restore leaving the tarball on disk after extraction
($src_cache was scoped to the previous PowerShell step)
- Bump src-cache key v1 -> v2
* ci: fetch llvm/rust toolchains in gn-check and clang-tidy
These workflows restore the src cache but don't run fix-sync. Now that
llvm-build and rust-toolchain are excluded from the cache, they need to
download them directly — gn gen read_file()s both, and clang-tidy runs
the binary from llvm-build.
* ci: fetch clang-tidy package explicitly
update.py's default 'clang' package doesn't include the clang-tidy
binary; it ships as a separate package.
* ci: preserve blink/web_tests/BUILD.gn when stripping test data
//BUILD.gn references //third_party/blink/web_tests:wpt_tests as a
target label, so the BUILD.gn must exist for gn gen. The data = [...]
entries it declares are runtime-only and not existence-checked at gen
time, so the actual test directories can still be removed.
* ci: compress src cache with zstd and drop gclient sync -vv
The src cache was an uncompressed tar (~16GB after exclusions). Switch
to zstd -T0 --long=30 for ~4x smaller transfer and multi-threaded
compression. Decompress on restore:
- Linux/macOS: zstd -d -c | tar -xf -
- Windows: zstd -d to an intermediate .tar, then the existing 7z
-snld20 extraction (preserves symlink handling)
All filename references updated .tar -> .tar.zst. -f added to the two
-o invocations so re-runs overwrite instead of failing.
Also drop -vv from gclient sync; default verbosity is sufficient.
* ci: keep .tar extension for src cache (zstd content inside)
The sas-sidecar that issues Azure SAS tokens validates filenames against
/^v[0-9]+-[a-z\-]+-[a-f0-9]+\.(tar|tgz)$/ and is not easily redeployed,
so keep the .tar extension and decode zstd on restore. Windows
decompresses to a distinct intermediate (src_cache.tar) so input and
output don't collide.
* ci: log NTFS 8.3/lastaccess/Defender state before Windows cache extract
Temporary diagnostics to see whether 8.3 short-name generation is the
cause of the ~20 min tar extraction.
* ci: revert src-cache exclusion additions
The new exclusions (web_tests contents, jetstream, speedometer,
catapult test_data, regres, llvm-build, rust-toolchain) caused siso/RBE
cache misses — even data-only deps are part of action input hashes.
Revert to the original exclusion list and drop the corresponding
toolchain-fetch plumbing. zstd compression, the Windows tar cleanup,
and the -vv removal remain.
* ci: drop win_toolchain from src cache; remove NTFS diagnostics
The Windows src cache includes 14.6GB of depot_tools/win_toolchain —
7.3GB of MSVC/SDK doubled because tar captures both the vs_files.ciopfs
backing store and the live ciopfs mount at vs_files/. Every Windows
cache consumer already re-fetches this via vs_toolchain.py update
--force (fix-sync for build/publish, inline for gn-check/clang-tidy),
so the cached copy is never used.
Diagnostics removed — CI confirmed 8dot3, last-access, and Defender are
all already off on the AKS Windows nodes.
* ci: unmount ciopfs vs_files before removing win_toolchain
vs_files is a live ciopfs mount during the win-targeted checkout; rm -rf
fails with EBUSY until it's unmounted.
* ci: skip win_toolchain download during checkout instead of removing after
fusermount isn't on the checkout container, so the ciopfs mount can't be
torn down before rm. Setting DEPOT_TOOLS_WIN_TOOLCHAIN=0 makes the
win_toolchain hook a no-op (vs_toolchain.py:525-527), so there's no
download and no mount. All Windows consumers re-fetch it post-restore
anyway. The rm -rf stays as a safety net.
* ci: also set ELECTRON_DEPOT_TOOLS_WIN_TOOLCHAIN=0 for checkout sync
build.yml sets ELECTRON_DEPOT_TOOLS_WIN_TOOLCHAIN=1 at the job level for
the Windows checkout, which makes e d inject DEPOT_TOOLS_WIN_TOOLCHAIN=1
and override the inline =0. Need both: the ELECTRON_ var stops e d from
overriding, the plain one stops vs_toolchain.py from defaulting to 1.
* ci: extract Windows src cache with piped tar instead of 7z
7z takes ~20 min to extract the ~1.1M-entry tar regardless of size —
~1ms per entry of header parsing and path handling, single-threaded,
well under the 75k IOPS / 1000 MBps the ephemeral disk can do. Switch
to the same zstd -d | tar -xf - pipe used on Linux/macOS (via Git Bash
tar). No intermediate src_cache.tar, download deleted after extract.
The -snld20 flag was working around 7z's own "dangerous symlink"
refusal; GNU tar extracts symlinks as-is so it shouldn't be needed.
* ci: keep depot_tools/win_toolchain scripts in src cache
The rm -rf removed get_toolchain_if_necessary.py (a depot_tools source
file), breaking vs_toolchain.py update --force on restore.
DEPOT_TOOLS_WIN_TOOLCHAIN=0 on the sync already prevents the vs_files
download, so the rm was only removing scripts.
* ci: split src cache into 4 parallel-extractable shards
Windows tar extraction is ~1ms/entry for ~1.2M entries (~20 min)
regardless of tool, well under the 75k IOPS / 1000 MBps the D16lds_v5
ephemeral disk can do. Tar is a sequential stream so the only way to
parallelize is to split at creation time.
Shards (balanced by entry count, ~220-360k each):
a: src/third_party/blink
b: src/third_party/{dawn,electron_node,tflite,devtools-frontend}
c: src/third_party (rest)
d: src (excluding third_party)
DEPSHASH is now the raw hash; shard files are
v2-src-cache-shard-{a..d}-${DEPSHASH}.tar (all pass the sas-sidecar
filename regex). sas-token is now a JSON keyed by shard letter. All
restore paths extract the four shards in parallel with per-PID wait so
a failed shard aborts the step.
* Revert "ci: split src cache into 4 parallel-extractable shards"
This reverts commit 970574998b.
fix-sync re-downloads llvm-build on macOS/Windows with the base clang
and objdump packages, but not clang-tidy. A local gclient sync pulls
clang-tidy (checkout_clang_tidy=True in DEPS), so CI's llvm-build tree
diverges from a local one. siso hashes the toolchain as action input,
so cache-only local runs against the CI-populated RBE cache miss.
Co-authored-by: trop[bot] <37223003+trop[bot]@users.noreply.github.com>
Co-authored-by: Samuel Attard <sattard@anthropic.com>
* remove comment based label removal
* ci: add functionality for programmatic add/remove needs-signed-commits label
* add new line to pull-request-opened-synchronized
* build: enable V8 builtins PGO
Removes the gn arg that disabled V8 builtins profile-guided optimization
and adds a V8 patch to warn instead of abort when the builtin PGO profile
data does not match. Also strips the PGO-related flags from the generated
mksnapshot_args so they are not passed through to downstream mksnapshot
invocations.
Co-authored-by: Sam Attard <sattard@anthropic.com>
* docs: clarify Node.js async_hooks as reason for promise_hooks flag
Addresses review feedback: the v8_enable_javascript_promise_hooks flag
is set to support Node.js async_hooks, not used directly by Electron.
Co-authored-by: Sam Attard <sattard@anthropic.com>
* chore: update patches
---------
Co-authored-by: trop[bot] <37223003+trop[bot]@users.noreply.github.com>
Co-authored-by: Sam Attard <sattard@anthropic.com>
Co-authored-by: PatchUp <73610968+patchup[bot]@users.noreply.github.com>
The update-patches artifact is a single .patch file, so zipping it
is unnecessary overhead. With archive: false, gh run download fetches
the raw file directly without requiring a decompression step.
Co-authored-by: trop[bot] <37223003+trop[bot]@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
ci: upload patch conflict fix as artifact in apply-patches
When patch-up.js cannot auto-push the 3-way-merged patch diff (e.g. on
fork PRs), the checkout action already writes patches/update-patches.patch
and tells the user to check CI artifacts — but nothing was uploading it.
This adds the missing upload-artifact step to the apply-patches job so
the resolved diff is available for download, and documents in CLAUDE.md
that pulling this artifact and applying it with `git am` is the fast
path for fixing patch conflicts on PR branches without a full local sync.
Co-authored-by: trop[bot] <37223003+trop[bot]@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
fix: authenticate curl requests to googlesource in lint workflow
The "Download GN Binary" and "Download clang-format Binary" steps
fetch files from chromium.googlesource.com without passing
authentication cookies. When googlesource rate-limits or returns a
transient error (502), the HTML error page is piped into `base64 -d`,
causing `base64: invalid input`.
The `set-chromium-cookie` action already configures `~/.gitcookies`
in a prior step. Pass `-b ~/.gitcookies` to both `curl` calls so
they authenticate, matching what the cookie verification step itself
does.
Co-authored-by: trop[bot] <37223003+trop[bot]@users.noreply.github.com>
Co-authored-by: Shelley Vohr <shelley.vohr@gmail.com>
