Skip to content

ci: reuse trusted CI container images when their context is unchanged - #7853

Merged
PastaPastaPasta merged 1 commit into
dashpay:developfrom
PastaPastaPasta:ci/reuse-trusted-ci-images
Oct 8, 2026
Merged

PastaPastaPasta merged 1 commit into
dashpay:developfrom
PastaPastaPasta:ci/reuse-trusted-ci-images

Conversation

@PastaPastaPasta

@PastaPastaPasta PastaPastaPasta commented Oct 7, 2026 •

Copy link
Copy Markdown
Member

Issue being fixed or feature implemented

build.yml calls build-container.yml twice, once for dashcore-ci-runner and once for dashcore-ci-slim. Each call runs build-amd64, build-arm64 and create-multi-arch-manifest on GitHub-hosted runners. Over the last 60 build.yml runs, each of these six jobs did 20–30 s of work but waited a p50 of 40–48 min (p90 55–81 min) for a hosted runner. Lint and every depends job need these jobs, so lint started a p50 of 94 min after a push, even though lint itself runs on self-hosted hardware with no wait.

The image context almost never changes. Only 5 develop commits since 2026-04 touched contrib/containers/ci/, so nearly every run rebuilds an image that already exists.

What was done?

When a trusted image built from the same context already exists, CI reuses it and skips the six build jobs.

Reuse key. check-skip builds the tag ctx-<tree>-<noble[:12]>-<recipe[:12]> from three inputs:

  • <tree>: the git tree id of contrib/containers/ci at the commit under test. This covers ci.Dockerfile, ci-slim.Dockerfile (which ci.Dockerfile pulls in through FROM ./ci-slim.Dockerfile) and everything else in the build context.
  • <noble>: the ubuntu:noble digest that check-skip already computes for the depends cache key.
  • <recipe>: a hash of build.yml and build-container.yml from the check-skip checkout. That checkout is the same revision GitHub runs for the build, so a change to how images are built (platforms, Dockerfile mapping, cache settings) also changes the key without anyone remembering to bump it.

Lookup (.github/workflows/resolve_ci_images.py, new step in check-skip):

  1. Reads the tree id with one call, GET /repos/{repo}/contents/contrib/containers?ref=<commit>. The commit is the PR head for pull_request_target and github.sha for push.
  2. Gets a GHCR pull token with GITHUB_TOKEN (falling back to anonymous) and fetches ghcr.io/<repo>/<image>:<tag>.
  3. Reuses the image only if the manifest is a multi-arch index containing both linux/amd64 and linux/arm64, and the sha256 of the body matches Docker-Content-Digest.
  4. Outputs a digest reference (the image pinned to sha256:…) per image, or empty.

The step uses only stdlib Python, so it runs on the small arm check-skip runner. It fails open: an unknown noble digest, any API or registry error, or a script crash just means the images are built as before. The step has continue-on-error and a 2 minute timeout. Each image falls back independently. Setting SKIP_CI_IMAGE_REUSE forces a build.

build-container.yml

  • New inputs: existing-image (optional; the digest reference of a trusted container already built from this context), context-tag (optional; the ctx-… tag that push builds add so later runs can find the container) and runs-on-existing (the runner for the stand-in job below).
  • When existing-image is set, both build jobs and the manifest job are skipped, and the path output becomes inputs.existing-image || jobs.create-manifest.outputs.path.
  • A one-step Use existing container job, running on the check-skip runner, runs instead of the build when existing-image is set. It is required because when every job of a called workflow is skipped, the caller job counts as skipped, and so does every job that needs: it. That would silently skip lint, depends, src and test. The fork experiment below shows this.
  • On push only, create-manifest also writes the ctx-… tag, next to the tags it already writes.

Runner routing for the existing jobs is unchanged.

Trust. check-skip still checks out and runs only base branch code. The PR head is only read as data through the contents API. The commit id and tree id are validated as hex before they go into a URL, and nothing from the PR reaches a shell or a workflow command. The invariant "only trusted events may write tags that other runs read" still holds: only push builds write ctx- tags, and PR builds still write only untrusted. A PR can therefore only cause its own run to reuse an image that a trusted run built from the identical context and recipe. Consumers still get a digest reference, never a tag.

Unchanged:

  • The trusted-<hashFiles> cache-from tags.
  • DOCKERFILE_HASH and the ccache, ctcache and cppcheck keys. These use hashFiles on the checked-out Dockerfiles, which are identical whenever the image is reused.
  • guix-build.yml, which builds its own image.
  • release_docker_hub.yml.
  • ci-msan.Dockerfile, which no workflow builds. Because it sits in the context directory, editing it now triggers a rebuild. That is unneeded but safe.

Behaviour changes worth knowing:

  • On a push that reuses an image, create-manifest doesn't run, so :latest, :<branch>, :<hashFiles> and the per-arch trusted-… tags are not moved. No workflow reads them; they matter only to people pulling, for example, :develop by hand.
  • Unpinned inputs (debian:bookworm-slim, ghcr.io/astral-sh/uv:latest, the devthefuture/dockerfile-x frontend) are not part of the key. Today the inline layer cache mostly freezes them too, but a rebuild could pick up a newer version. Now they only refresh when the context, the recipe or the noble digest changes.
  • PR builds still run with a packages: write token while building a PR-controlled Dockerfile. This PR doesn't change that, but if that token ever leaked, a forged ctx- tag would now be used as the CI image itself, not just as build cache.

How Has This Been Tested?

flake8 (with the repo's enabled codes), mypy, lint-files.py, lint-python-utf8-encoding.py, lint-whitespace.py and actionlint pass on the touched files; actionlint reports only findings that were already there.

On the fork PastaPastaPasta/dash, with push events and no RUNNER_* vars:

The (a)–(c) runs were cancelled once the relevant jobs had run. I did not exercise pull_request_target in CI, because the fork sets SKIP_ON_PR. Instead I ran the script locally against the payload of a real cross-fork PR on this repository. It read the head's contrib/containers/ci tree through the base repository's contents API, and the result matched git rev-parse <head>:contrib/containers/ci.

Not verifiable from a fork: ghcr.io/dashpay/dash/dashcore-ci-runner is private (anonymous token requests get 401, while dashcore-ci-slim is public). On this repository, the lookup for the runner image therefore depends on check-skip's GITHUB_TOKEN (packages: read) being able to read it. If it can't, that image just keeps being built, and the step summary says why.

Breaking Changes

None. Consumers get the same kind of digest reference as before. Any failure in the new lookup falls back to building the images exactly as before.

Checklist:

  • I have performed a self-review of my own code
  • I have made corresponding changes to the documentation
  • I have assigned this pull request to a milestone (for repository code-owners and collaborators only)

🤖 Generated with Claude Code

@thepastaclaw

thepastaclaw commented Oct 7, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Final review complete — no blockers (commit f7dd729)

@PastaPastaPasta
PastaPastaPasta force-pushed the ci/reuse-trusted-ci-images branch from c53cc42 to e3d0aba Compare October 7, 2026 20:25
Every CI run rebuilt dashcore-ci-runner and dashcore-ci-slim: six jobs
that each do 20-30 s of work but wait a p50 of 40-48 min for a hosted
runner, which holds lint and every depends job back. The build context
rarely changes, so almost every run rebuilds an image that already
exists.

check-skip now derives a reuse tag from the git tree id of
contrib/containers/ci at the commit under test, the ubuntu:noble digest
and a hash of the workflows that drive the build, and looks it up in
GHCR. When a trusted image with that tag exists, the container workflow
skips its build jobs and hands the image's digest reference to the
dependent jobs instead.

- The commit under test is read only as data through the contents API.
  check-skip still checks out and runs only base branch code.
- Only push builds publish the reuse tag, next to the tags they already
  write, so a reused image was always built by a trusted run. Pull
  request builds still only write `untrusted`.
- When reusing, one short job on the check-skip runner stands in for
  the build. If every job of a called workflow is skipped, the caller
  counts as skipped and so does every job that needs it.
- Any lookup failure, or an unknown base image digest, builds as before.
  SKIP_CI_IMAGE_REUSE forces a build.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@PastaPastaPasta
PastaPastaPasta force-pushed the ci/reuse-trusted-ci-images branch from e3d0aba to f7dd729 Compare October 7, 2026 20:27
@coderabbitai

coderabbitai Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 5320a282-976a-43d0-b499-2fd23b325574
📥 Commits

Reviewing files that changed from the base of the PR and between 2340e3a and f7dd729.

📒 Files selected for processing (3)
  • .github/workflows/build-container.yml
  • .github/workflows/build.yml
  • .github/workflows/resolve_ci_images.py

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 2 remain after this review.


Walkthrough

The build workflow resolves reusable CI container images from the tested commit context and registry. It passes resolved image references and a context tag to the container workflows. When an existing image is supplied, the container workflow reports its digest and skips building. Otherwise, it builds the architecture images and manifest. Trusted builds can include the context tag in their published tags.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant build.yml
  participant resolve_ci_images.py
  participant GitHubAPI
  participant ContainerRegistry
  participant build-container.yml
  build.yml->>resolve_ci_images.py: Pass event, base digest, and reuse setting
  resolve_ci_images.py->>GitHubAPI: Read tested commit context tree
  GitHubAPI-->>resolve_ci_images.py: Return context tree
  resolve_ci_images.py->>ContainerRegistry: Request token and image indexes
  ContainerRegistry-->>resolve_ci_images.py: Return image indexes
  resolve_ci_images.py-->>build.yml: Return image references and context tag
  build.yml->>build-container.yml: Pass existing image and context tag
  alt Existing image is supplied
    build-container.yml-->>build.yml: Report existing image digest
  else No existing image is supplied
    build-container.yml->>ContainerRegistry: Publish built architecture images and manifest
  end
Loading

Merge Risk: ⚪ Minimal · up to f7dd7

No actionable merge-blocking issue remains after normal checks. Private registry lookup remains unverified, but a lookup failure falls back to building.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 38.46% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 13 functions across 1 files. (2 skipped: … Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and concisely describes the main change: reusing trusted CI container images when their build context is unchanged.
Description check ✅ Passed The description directly explains the CI image reuse design, trust model, workflow changes, testing, and behavior changes.
Full details: Docstring Coverage

Explanation

Docstring coverage is 38.46% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 13 functions across 1 files. (2 skipped: 2 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final validation — Phase 1 + Phase 2

Verified all supplied findings against head f7dd729 and the surrounding container workflows. No blocking defect was confirmed; the overlapping test-coverage suggestions are consolidated below, and the speculative naming-coupling nitpick is dropped. Validation was static only: the supplied CI snapshot shows check-skip passing, with container builds and other checks queued or pending.

🟡 1 suggestion(s)

Review provenance

Source: reviewer 1: muse-spark-1.3-contributor (agent: phase1-reviewer, role: general); reviewer 2: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 3: gpt-6.1-sol (agent: phase2-reviewer, role: dash-core-commit-history); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)

  • Triage: normal by gpt-6.1-sol (effort low) — The change adds nontrivial CI image resolution, trust validation, and workflow skip behavior across three files, but does not modify a critical surface as defined.
  • Phase 1 reviewers: muse-spark-1.3-contributor — general (completed, effort xhigh); agent phase1-reviewer
  • Phase 1 model: muse-spark-1.3-contributor — not quota-gated; passed over gemini-3.8-flash-high (antigravity below 15% reserve: weekly 15% left, 5h 100% left), glm-5.3-flash (not used above high effort; tier asks max)
  • Fresh verifier: gpt-6.1-sol — final-verifier; agent sol-verifier
  • Phase 2 reviewers: gpt-6.1-sol — general (completed, effort high); agent phase2-reviewer, gpt-6.1-sol — dash-core-commit-history (completed, effort high); agent phase2-reviewer
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.

In `.github/workflows/resolve_ci_images.py`:
- [SUGGESTION] .github/workflows/resolve_ci_images.py:218-231: Add deterministic coverage for image rejection and independent fallback
  The new resolver has no automated regression tests, although its injected `get` callable makes these decisions testable without registry access. The documented fork runs exercise both images hitting, both missing, and context invalidation, but not rejection of an unsuitable manifest or one image failing while the other remains reusable. These outputs directly control whether the container build jobs execute. Add a sibling unittest module following `test_select_dynamic_runner.py`, with a valid multi-arch index as the positive case and cases for a digest mismatch, a missing required architecture, an unsupported media type, and a registry failure affecting only one image. Also assert that `pull_request_target` queries the PR head rather than the base SHA, and that unknown base digests or disabled reuse leave image outputs empty.

Comment on lines +218 to +231
for key, name in IMAGES:
image_path = "{}/{}".format(repo.lower(), name)
try:
registry = registry_token(get, image_path, token, actor)
digest = multi_arch_digest(get, image_path, tag, registry)
except Exception as exc: # noqa: BLE001
if isinstance(exc, HttpError) and exc.status == 404:
reason = "build: no trusted image tagged {}".format(tag)
else:
reason = "build: registry lookup failed ({}: {})".format(type(exc).__name__, exc)
outputs["{}_reason".format(key)] = reason
else:
outputs["{}_image".format(key)] = "{}/{}@{}".format(REGISTRY, image_path, digest)
outputs["{}_reason".format(key)] = "reuse: trusted image tagged {}".format(tag)

@thepastaclaw thepastaclaw Oct 7, 2026 •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⏭️ Deferred (maintainer decision; not blocking) at f7dd7291; see the replies below.

🟡 Suggestion: Add deterministic coverage for image rejection and independent fallback

The new resolver has no automated regression tests, although its injected get callable makes these decisions testable without registry access. The documented fork runs exercise both images hitting, both missing, and context invalidation, but not rejection of an unsuitable manifest or one image failing while the other remains reusable. These outputs directly control whether the container build jobs execute. Add a sibling unittest module following test_select_dynamic_runner.py, with a valid multi-arch index as the positive case and cases for a digest mismatch, a missing required architecture, an unsupported media type, and a registry failure affecting only one image. Also assert that pull_request_target queries the PR head rather than the base SHA, and that unknown base digests or disabled reuse leave image outputs empty.

source: muse-spark-1.3-contributor (phase1-reviewer: general); gpt-6.1-sol (phase2-reviewer: general)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Declining this one intentionally: the maintainer chose not to carry a dedicated unit-test module for this resolver (an earlier revision had one and it was removed on purpose). The behaviour that gates the build jobs is fail-open — any rejection, lookup error or unknown digest just means the images get built as before — and the hit / miss / context-change paths are covered by the fork runs linked in the description.


🤖 Posted autonomously by Claude on behalf of pasta.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Understood; I checked the per-image exception handling and empty-output build fallback, and I accept your deliberate choice not to carry a dedicated test module as a non-blocking coverage trade-off.

Noted as intentionally deferred; I will not press it further here.

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review after discussion — commit f7dd729

Standing review was COMMENTED; this re-review is APPROVE: no blocking findings remain.

Approved: the discussion withdrew or deferred every finding the full review of f7dd7291 (run 3985) raised, and the approval rests on that review.

Same commit as the standing review; the inline threads above carry the per-finding outcome. This follow-up exists only to correct the verdict.

Review provenance

Source: conversation lane gpt-6.1-sol (agent: conversation); no reviewer or verifier lanes ran for this follow-up

  • Verdict moved because the inline discussion withdrew or deferred findings of the standing review of this commit; the code was not re-reviewed

@PastaPastaPasta
PastaPastaPasta merged commit 28de0ec into dashpay:develop Oct 8, 2026
47 of 51 checks passed
@PastaPastaPasta
PastaPastaPasta deleted the ci/reuse-trusted-ci-images branch October 8, 2026 14:58
PastaPastaPasta added a commit that referenced this pull request Oct 8, 2026
… runners are backed up

8cfa2ac ci: keep push container builds on GitHub-hosted runners (pasta)
099cd2c ci: run pre-lint jobs on self-hosted runners when hosted runners are backed up (pasta)

Pull request description:

  ## Issue being fixed or feature implemented

  When GitHub-hosted runners are backed up, `build.yml` already sends the Lint job to our self-hosted runners (`vars.RUNNER_SELFHOSTED`, label `ubuntu-core`). The short jobs that lint and the long builds wait on still sit in the hosted queue. Over the last 60 runs, cache-sources, container build-amd64/build-arm64 and create-manifest waited a p50 of 20–48 min and a p90 of 55–81 min in the hosted queue, for 10–30 s of work each.

  #7853 skips most container builds by reusing trusted images. When the CI image context changes, though, those builds still run and still queue, and cache-sources queues on every run.

  ## What was done?

  The lint rung becomes a **pre-lint rung** that covers all of these jobs.

  `select_dynamic_runner.py`:
  - The `runner_lint` output is renamed to `runner_prelint` (amd64, label from `RUNNER_SELFHOSTED`), and `lint_decision_reason` to `prelint_decision_reason`.
  - New output `runner_prelint_arm64`, label from a new optional variable `RUNNER_SELFHOSTED_ARM64`, with its reason in `prelint_arm64_decision_reason`. If the variable is unset, arm64 pre-lint jobs keep `runner_arm64` with reason `selfhosted-arm64-disabled`. No arm64 self-hosted runner exists yet, so this PR changes nothing for arm64 until the variable is set.
  - The decision is the same as before, per arch: any queued GitHub-hosted job across dash/platform/grovedb/rust-dashcore; actor, PR author and head-repo owner all in `SELFHOSTED_LINT_AUTHORS` (push events: actor only); no `blacksmith-ci` label. Otherwise the job keeps that arch's existing hosted/Blacksmith decision.
  - **Saturation guard:** the existing jobs scan now also counts queued jobs per self-hosted label. If more than `SELFHOSTED_MAX_QUEUED` (default 6) are waiting for a label, that arch falls back to the hosted ladder with reason `selfhosted-saturated:queued:N>6`. The scan makes no extra API calls, and self-hosted jobs are still excluded from the hosted backlog figure that drives Blacksmith.
  - The step summary shows both pre-lint runners and both reasons. Successful reasons include the self-hosted queue depth, e.g. `selfhosted:backlog:3>0;queued:1<=6`.

  Routing in `build.yml`:

  | Job | Before | After |
  | --- | --- | --- |
  | Lint | `runner-lint` | `runner-prelint` |
  | Cache depends sources | `runner-arm64` | `runner-prelint` (the cache key has no runner in it, so any arch can fill it) |
  | Container build (amd64), both images | `runner-amd64` | pull requests: `runner-prelint`; pushes: `runner-amd64` (unchanged) |
  | Container build (arm64), both images | `runner-arm64` | pull requests: `runner-prelint-arm64`; pushes: `runner-arm64` (unchanged) |
  | Create multi-arch manifest, both images | `runner-arm64` | pull requests: `runner-prelint`; pushes: `runner-arm64` (unchanged). Set through a new `runs-on-manifest` input on `build-container.yml` |
  | Depends, src builds, tests | unchanged | unchanged |
  | Use existing container (#7853) | check-skip runner | unchanged |

  **Trust gate.** Every routed job except create-manifest runs pull request head code: the lint scripts, `make -C depends download`, and the Dockerfile `RUN` steps. All of them therefore go through the same allowlist that lint uses today. This PR does not widen who can reach the self-hosted runners.

  **Pushes keep their container builds on GitHub's runners.** Push builds publish the `trusted-*` images that #7853 reuses on later runs, so they never run on the persistent self-hosted runner, where earlier allowlisted jobs have run. Push builds are also not waited on by a reviewer, so the queue time matters less there. Lint and cache-sources on pushes still use the pre-lint rung, gated on the actor being in the allowlist.

  **Kill switch.** Unset `RUNNER_SELFHOSTED` to send all amd64 pre-lint jobs back to the hosted/Blacksmith ladder; unset `RUNNER_SELFHOSTED_ARM64` (or leave it unset) to do the same for arm64. No repository variables are changed by this PR.

  **Deployment note.** `actions/cache` includes the compression method in the cache version. The `ubuntu-core` runner host needs `zstd` installed, or cache-sources would save a gzip entry that the depends jobs (which restore with zstd inside the CI container) cannot read.

  **CI image rebuild.** #7853's context tag includes a hash of `build.yml` and `build-container.yml`, so merging this PR changes that hash and causes one rebuild of the CI images on the next trusted push.

  ## How Has This Been Tested?

  - `python3 .github/workflows/test_select_dynamic_runner.py`: 30 tests pass. The existing tests were updated for the renamed outputs, and new cases cover the arm64 rung (disabled, enabled, allowlist) and the saturation guard (over the cap, exactly at the cap, counted per label).
  - flake8 with the codes enabled in `test/lint/lint-python.py`: clean. `lint-python-utf8-encoding.py`, `lint-whitespace.py` and `lint-files.py`: clean.
  - Pushes are routed by expressions in `build.yml` (`github.event_name == 'push' && <hosted output> || <pre-lint output>`); both branches are non-empty runner labels, so the `&&`/`||` form behaves as a conditional.
  - actionlint on `build.yml`, `build-container.yml`, `cache-depends-sources.yml` and `lint.yml` reports the same 11 findings as on develop with #7853 merged (`allow-unsafe-pr-checkout` and existing shellcheck notes), none new.
  - R730 runner smoke test (`ubuntu-core` host, CT 161: Docker 29.8.1, buildx v0.37.1, 32 cores): as the `runner` user, cloned this branch, ran `docker buildx create --driver docker-container`, then `docker buildx build --platform linux/amd64 -f contrib/containers/ci/ci-slim.Dockerfile contrib/containers/ci`. The docker-container builder started and the build was running normally (the cppcheck stage was compiling) when SSH access to the host dropped. The final result and timing are not recorded yet; this description will be updated with them.

  ## Breaking Changes

  None for users. CI-internal changes:
  - The check-skip output `runner-lint` is renamed to `runner-prelint`.
  - `build-container.yml` has a new required input, `runs-on-manifest`. Its only callers, in `build.yml`, are updated.
  - With the rung off, cache-sources (all events) and create-manifest (pull requests only) run on `runner-amd64` instead of `runner-arm64`.

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have made corresponding changes to the documentation
  - [ ] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

  🤖 Generated with [Claude Code](https://claude.com/claude-code)

Top commit has no ACKs.

Tree-SHA512: 1950eb19e93f20ae21e7ea40bcb473c507298a18df7d8daedc4355467b8b44d13b0926d9882462b6d9d318b8813a785105a189657e5334cce41115c0d2f2cdf9
PastaPastaPasta added a commit that referenced this pull request Oct 15, 2026
…ions

3851103 lint: add zizmor GitHub Actions linter (pasta)
2fc8e33 ci: move untrusted expressions into env and tighten workflow permissions (pasta)

Pull request description:

  ## Issue being fixed or feature implemented
  Dash Core has no linter for its GitHub Actions workflows. Running [zizmor](https://docs.zizmor.sh) 1.30.1 offline on current `develop` reports 145 unsuppressed findings, including 32 `template-injection` (22 high), 7 `excessive-permissions`, 6 `dangerous-triggers`, 4 `unpinned-images`, 4 `artipacked`, 26 `self-repository` and 66 `unpinned-uses`.

  An earlier version of this PR also stopped pull request runs from pushing the shared `dashcore-ci-{runner,slim}:latest` / base-branch tags and built the container manifests from this run's image digests. #7809 has since merged and covers both: untrusted runs only write `:untrusted` tags, and manifests are assembled from the per-arch digests. Those two commits were dropped, and this PR is now rebased on `develop` after #7809, #7853 and #7854.

  ## What was done?
  Two commits:

  1. **`ci: move untrusted expressions into env and tighten workflow permissions`**. `${{ }}` expressions used inside `run:` scripts (and the release workflow's `github-script` body) are now passed through step `env:` entries and read as quoted shell variables. Where the runner already provides the value (`GITHUB_EVENT_NAME`, `GITHUB_REF`, `GITHUB_REF_NAME`, `GITHUB_SHA`, `GITHUB_WORKSPACE`), the built-in variable is used directly. The workflows do the same thing as before:
     - `DEP_OPTS` keeps its shell-quoting semantics. It is split with `eval "DEP_OPTS_ARGS=(${DEP_OPTS})"`, so quoted values such as tsan's `CXX='clang++-19 -stdlib=libc++'` and the msan `CFLAGS`/`CXXFLAGS` produce exactly the same arguments as the old inline substitution. A plain `env ${DEP_OPTS}` would not.
     - `build-container.yml`'s manifest step (as rewritten by #7809/#7853) reads the repo, image name, tags and per-arch digests from `env:`.
     - `guix-build.yml` passes the digest-pinned builder image (`needs.build-image.outputs.image`, from #7809) and the build target through `env:`.
     - `BASE_OUTDIR="/output"` in `build-src.yml` is left alone. It is pre-existing, `develop` no longer sets it in the unit-test step, and this PR does not touch it.
     - Least-privilege `permissions:` are added to `cache-depends-sources`, `clang-diff-format`, `prevent-master-pr`, `release_docker_hub` and `semantic-pull-request`.
     - `guix-build`'s workflow-level permissions become `{}`. `build-image` keeps the `packages: write` that #7809 gave it, and the `build` job gets `packages: read` plus the `id-token`/`attestations: write` it needs for provenance attestation.
     - `persist-credentials: false` is added to the remaining checkouts.
  2. **`lint: add zizmor GitHub Actions linter`**. `test/lint/lint-github-actions.py` runs `zizmor --offline` over `.github/workflows/` and is picked up by `all-lint.py`. Like the shellcheck and codespell linters, it skips when zizmor is not installed. zizmor 1.30.1 is pinned in `ci-slim.Dockerfile` and `ci/lint/04_install.sh`, alongside the other Python lint tools. Configuration lives in `.github/zizmor.yml`, and no ignore is keyed by line number:
     - `unpinned-uses`: policy `"*": ref-pin`. Actions stay pinned to release tags. SHA-pinning all 66 `uses:` is left for a separate decision. Every `uses:` must still name an explicit ref.
     - `self-repository` (new in zizmor 1.30): ignored for `build.yml` only. Its `./.github/workflows/...` references are local reusable workflows, which GitHub loads from the calling commit rather than from the runner filesystem.
     - `dangerous-triggers`: inline `# zizmor: ignore[...]` with a reason on each of the six `pull_request_target` triggers. Each trigger is needed for fork PRs.
     - `unpinned-images`: inline ignore on the four CI container jobs, whose image is built (or reused by digest) earlier in the same run.

  `test/util/data/non-backported.txt` is not updated because it only lists C/C++ files for clang-format.

  ## How Has This Been Tested?
  - zizmor 1.30.1 `--offline` on the branch reports "No findings to report" (36 ignored, 52 suppressed). The workflow code added by #7809, #7853 and #7854 needed no new ignores.
  - `test/lint/lint-github-actions.py`:
    - passes locally (macOS);
    - exits 0 with a skip message when zizmor is absent;
    - exits 1 when a `${{ github.event.pull_request.title }}` is injected into a `run:` step.
  - `actionlint` 1.7.12 (with shellcheck) on all workflows: no new findings versus `develop`, and one pre-existing shellcheck warning (`merge-check.yml` SC2193) goes away. Only the checkout v6 `allow-unsafe-pr-checkout` input is unknown to actionlint, on both branches.
  - The `DEP_OPTS` array expansion was checked against the old literal substitution for `linux64_tsan`, `linux64_msan`, `linux64_multiprocess`, `linux64` and `mac`. The arguments are identical. That code is unchanged by the rebase.
  - `lint-whitespace.py` and `lint-files.py` pass on each commit. `lint-python-utf8-encoding.py`, `lint-python-dead-code.py`, `lint-python-mutable-default-parameters.py`, `lint-shell.py` and flake8 on the new script pass.
  - CI evidence: `pull_request_target` runs use the workflow files from `develop`, so this PR's own dashpay/dash run does not exercise the edited workflows. The `release` and `guix-build` changes are covered by review only.

  ## Breaking Changes
  None.

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have made corresponding changes to the documentation
  - [ ] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

  🤖 Generated with [Claude Code](https://claude.com/claude-code)

Top commit has no ACKs.

Tree-SHA512: b26f474b79af9b92bbc0304867c0d90dd6811e43c9efa053798c4c04245919bd63fb4ff96c6adfcda14785c5973e75af0f9e6cc87e243366796d2bc5fb71ea69
@UdjinM6 UdjinM6 added this to the 24.1 milestone Oct 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants