CI perf: ci.yml test-release — single 29.5-min job is now the whole PR critical path (~12 min off PR push→ci-ok) #1181

Measurement

Window: PR CI runs created 2026-10-08 19:00–20:40 UTC, after #1133 (00c658f) reached PR heads. Jobs API sample: 14 successful non-draft PR runs.

• PR head push → ci-ok: p50 29.0 min, p90 30.3 min.
• The last job to finish was test-release in every post-Cut merge-group CI from ~46 to ~20 min: shard Gradle e2e and test legs, skip test-release in queue, cancel orphaned runs #1133 PR run with the full job set, e.g. https://github.com/SocketDev/socket-patch/actions/runs/37834457136/job/113507817687
• test-release run time p50 29.5 min (n=9); 7-day p50 26.8 / p90 29.9 (n=144).
• The next-longest PR job is test (windows-latest, 2) at ~16.6 min, then coverage ~15.7 min. e2e Gradle legs, which bounded PR latency at ~33 min before Cut merge-group CI from ~46 to ~20 min: shard Gradle e2e and test legs, skip test-release in queue, cancel orphaned runs #1133, are now ~10–13 min.
• test-release is already skipped in merge_group (Cut merge-group CI from ~46 to ~20 min: shard Gradle e2e and test legs, skip test-release in queue, cancel orphaned runs #1133), so it only affects PR feedback latency and push-to-main runs (20–30 min there).

Where the time goes

Per the job's own comment, ~23 min of it is compiling and linking ~240 optimized test binaries (ci-release = release minus LTO); the test run itself is ~6 min.

Root cause

#1133 split test into two shards per OS with scripts/ci-test-shard.py (one shard links about half of the ~240 integration targets) but left test-release as one job, so it's now ~13 min longer than anything else on the PR path.

Proposed fix

In .github/workflows/ci.yml test-release (ci.yml:414): add strategy.matrix.shard: [1, 2] (or [1, 2, 3]) and select targets with the same scripts/ci-test-shard.py the test job uses, e.g. cargo test --workspace --profile ci-release $(python3 scripts/ci-test-shard.py --shard ${{ matrix.shard }} …). Each shard compiles the shared deps from the rust-cache, then links only its own targets. Job name becomes test-release (1) / test-release (2). ci-ok's needs names the job id, so it keeps working. Check that ci-ok's result evaluation treats a matrix job the same way it does for test.

Optionally also save a ci-release rust-cache key on main (save-if: main, key: ci-release) if the cache restored for this job is the dev-profile one. Check the "Cache cargo" restore key in a log first.

Expected saving

• PR push → ci-ok: ~29 → ~17–18 min p50 (≈ −11 to −12 min), then bounded by test (windows-latest, 2) (~16.6 min) and coverage (~15.7 min). Across ~180 PR CI runs/day.
• Push-to-main CI: 20–30 → ~15 min.
• Cost: +1 Linux job per PR/push run, re-restoring the cache and re-compiling non-cached workspace crates (~3–5 min) ≈ +600–900 Linux job-min/day. macOS: 0.

Coverage and risk

The same tests run in release profile on every PR and push to main; only the partitioning changes. ci-test-shard.py already guarantees every target lands in exactly one shard for test. Required checks ci-ok / clippy are unchanged (test-release isn't required directly, and ci-ok already counts a skipped test-release as passing in merge_group).

Effort

S (copy the test job's shard plumbing).

ROI

~12 min × 180 PR runs/day of feedback latency (latency-weighted ≈ 2,000 pts) × confidence 0.8 / effort 1 ≈ 1600.