mirror of
https://github.com/electron/electron.git
synced 2026-04-10 03:01:51 -04:00
fef2fd2941
* ci: shrink src cache and fix Windows tar cleanup
- Exclude platform-specific toolchains (llvm-build, rust-toolchain) from
the src cache; all platforms now fetch them via fix-sync post-restore
- Exclude unused test data and benchmarks: blink/web_tests, jetstream,
speedometer, catapult/tracing/test_data, swiftshader/tests/regres
- Fix Windows restore leaving the tarball on disk after extraction
($src_cache was scoped to the previous PowerShell step)
- Bump src-cache key v1 -> v2
* ci: fetch llvm/rust toolchains in gn-check and clang-tidy
These workflows restore the src cache but don't run fix-sync. Now that
llvm-build and rust-toolchain are excluded from the cache, they need to
download them directly — gn gen read_file()s both, and clang-tidy runs
the binary from llvm-build.
* ci: fetch clang-tidy package explicitly
update.py's default 'clang' package doesn't include the clang-tidy
binary; it ships as a separate package.
* ci: preserve blink/web_tests/BUILD.gn when stripping test data
//BUILD.gn references //third_party/blink/web_tests:wpt_tests as a
target label, so the BUILD.gn must exist for gn gen. The data = [...]
entries it declares are runtime-only and not existence-checked at gen
time, so the actual test directories can still be removed.
* ci: compress src cache with zstd and drop gclient sync -vv
The src cache was an uncompressed tar (~16GB after exclusions). Switch
to zstd -T0 --long=30 for ~4x smaller transfer and multi-threaded
compression. Decompress on restore:
- Linux/macOS: zstd -d -c | tar -xf -
- Windows: zstd -d to an intermediate .tar, then the existing 7z
-snld20 extraction (preserves symlink handling)
All filename references updated .tar -> .tar.zst. -f added to the two
-o invocations so re-runs overwrite instead of failing.
Also drop -vv from gclient sync; default verbosity is sufficient.
* ci: keep .tar extension for src cache (zstd content inside)
The sas-sidecar that issues Azure SAS tokens validates filenames against
/^v[0-9]+-[a-z\-]+-[a-f0-9]+\.(tar|tgz)$/ and is not easily redeployed,
so keep the .tar extension and decode zstd on restore. Windows
decompresses to a distinct intermediate (src_cache.tar) so input and
output don't collide.
* ci: log NTFS 8.3/lastaccess/Defender state before Windows cache extract
Temporary diagnostics to see whether 8.3 short-name generation is the
cause of the ~20 min tar extraction.
* ci: revert src-cache exclusion additions
The new exclusions (web_tests contents, jetstream, speedometer,
catapult test_data, regres, llvm-build, rust-toolchain) caused siso/RBE
cache misses — even data-only deps are part of action input hashes.
Revert to the original exclusion list and drop the corresponding
toolchain-fetch plumbing. zstd compression, the Windows tar cleanup,
and the -vv removal remain.
* ci: drop win_toolchain from src cache; remove NTFS diagnostics
The Windows src cache includes 14.6GB of depot_tools/win_toolchain —
7.3GB of MSVC/SDK doubled because tar captures both the vs_files.ciopfs
backing store and the live ciopfs mount at vs_files/. Every Windows
cache consumer already re-fetches this via vs_toolchain.py update
--force (fix-sync for build/publish, inline for gn-check/clang-tidy),
so the cached copy is never used.
Diagnostics removed — CI confirmed 8dot3, last-access, and Defender are
all already off on the AKS Windows nodes.
* ci: unmount ciopfs vs_files before removing win_toolchain
vs_files is a live ciopfs mount during the win-targeted checkout; rm -rf
fails with EBUSY until it's unmounted.
* ci: skip win_toolchain download during checkout instead of removing after
fusermount isn't on the checkout container, so the ciopfs mount can't be
torn down before rm. Setting DEPOT_TOOLS_WIN_TOOLCHAIN=0 makes the
win_toolchain hook a no-op (vs_toolchain.py:525-527), so there's no
download and no mount. All Windows consumers re-fetch it post-restore
anyway. The rm -rf stays as a safety net.
* ci: also set ELECTRON_DEPOT_TOOLS_WIN_TOOLCHAIN=0 for checkout sync
build.yml sets ELECTRON_DEPOT_TOOLS_WIN_TOOLCHAIN=1 at the job level for
the Windows checkout, which makes e d inject DEPOT_TOOLS_WIN_TOOLCHAIN=1
and override the inline =0. Need both: the ELECTRON_ var stops e d from
overriding, the plain one stops vs_toolchain.py from defaulting to 1.
* ci: extract Windows src cache with piped tar instead of 7z
7z takes ~20 min to extract the ~1.1M-entry tar regardless of size —
~1ms per entry of header parsing and path handling, single-threaded,
well under the 75k IOPS / 1000 MBps the ephemeral disk can do. Switch
to the same zstd -d | tar -xf - pipe used on Linux/macOS (via Git Bash
tar). No intermediate src_cache.tar, download deleted after extract.
The -snld20 flag was working around 7z's own "dangerous symlink"
refusal; GNU tar extracts symlinks as-is so it shouldn't be needed.
* ci: keep depot_tools/win_toolchain scripts in src cache
The rm -rf removed get_toolchain_if_necessary.py (a depot_tools source
file), breaking vs_toolchain.py update --force on restore.
DEPOT_TOOLS_WIN_TOOLCHAIN=0 on the sync already prevents the vs_files
download, so the rm was only removing scripts.
* ci: split src cache into 4 parallel-extractable shards
Windows tar extraction is ~1ms/entry for ~1.2M entries (~20 min)
regardless of tool, well under the 75k IOPS / 1000 MBps the D16lds_v5
ephemeral disk can do. Tar is a sequential stream so the only way to
parallelize is to split at creation time.
Shards (balanced by entry count, ~220-360k each):
a: src/third_party/blink
b: src/third_party/{dawn,electron_node,tflite,devtools-frontend}
c: src/third_party (rest)
d: src (excluding third_party)
DEPSHASH is now the raw hash; shard files are
v2-src-cache-shard-{a..d}-${DEPSHASH}.tar (all pass the sas-sidecar
filename regex). sas-token is now a JSON keyed by shard letter. All
restore paths extract the four shards in parallel with per-PID wait so
a failed shard aborts the step.
* Revert "ci: split src cache into 4 parallel-extractable shards"
This reverts commit 970574998b.
66 lines
2.0 KiB
YAML
66 lines
2.0 KiB
YAML
name: Electron Docs Compile
|
|
|
|
on:
|
|
workflow_call:
|
|
inputs:
|
|
container:
|
|
required: true
|
|
description: 'Container to run the docs-only ts compile in'
|
|
type: string
|
|
|
|
permissions: {}
|
|
|
|
concurrency:
|
|
group: electron-docs-only-${{ github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
env:
|
|
GCLIENT_EXTRA_ARGS: --custom-var=checkout_arm=True --custom-var=checkout_arm64=True
|
|
|
|
jobs:
|
|
docs-only:
|
|
name: Docs Only Compile
|
|
runs-on: electron-arc-centralus-linux-amd64-4core
|
|
permissions:
|
|
contents: read
|
|
timeout-minutes: 20
|
|
container: ${{ fromJSON(inputs.container) }}
|
|
steps:
|
|
- name: Checkout Electron
|
|
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
|
|
with:
|
|
path: src/electron
|
|
fetch-depth: 0
|
|
ref: ${{ github.event.pull_request.head.sha }}
|
|
- name: Generate DEPS Hash
|
|
run: |
|
|
node src/electron/script/generate-deps-hash.js
|
|
DEPSHASH=v2-src-cache-$(cat src/electron/.depshash)
|
|
echo "DEPSHASH=$DEPSHASH" >> $GITHUB_ENV
|
|
echo "CACHE_PATH=$DEPSHASH.tar" >> $GITHUB_ENV
|
|
- name: Restore src cache via AKS
|
|
uses: ./src/electron/.github/actions/restore-cache-aks
|
|
with:
|
|
target-platform: linux
|
|
- name: Checkout Electron
|
|
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
|
|
with:
|
|
path: src/electron
|
|
fetch-depth: 0
|
|
ref: ${{ github.event.pull_request.head.sha }}
|
|
- name: Install Dependencies
|
|
uses: ./src/electron/.github/actions/install-dependencies
|
|
- name: Run TS/JS compile
|
|
shell: bash
|
|
run: |
|
|
cd src/electron
|
|
node script/yarn.js create-typescript-definitions
|
|
node script/yarn.js tsc -p tsconfig.default_app.json --noEmit
|
|
for f in build/webpack/*.js
|
|
do
|
|
out="${f:29}"
|
|
if [ "$out" != "base.js" ]; then
|
|
node script/yarn.js webpack --config $f --output-filename=$out --output-path=./.tmp --env mode=development
|
|
fi
|
|
done
|