Repository navigation
Publish a daily GPU Operator chart and OLM bundle with main-branch operand images - #3087
Open
abrarshivani wants to merge 4 commits into
Open
abrarshivani wants to merge 4 commits into
abrarshivani wants to merge 4 commits into
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info
📝 Walkthrough
Priority: ➖ Normal Merge Risk: ⚪ Minimal · up to
|
Contributor
|
Instead of |
abrarshivani
force-pushed
the
oci-chart-operands-main
branch
3 times, most recently
from
October 16, 2026 22:46
b5d66fe to
11afc6b
Compare
get-latest-images.sh only knew toolkit, device-plugin and mig-manager, always read the main branch and looked up one component per call. Add driver-manager, vgpu-device-manager, the kata and kubevirt sandbox device plugins and cc-manager. Each component names the branch to read (main, or master for kubevirt-gpu-device-plugin) and an optional tag suffix, since k8s-cc-manager publishes only <sha>-distroless commit images. The script now takes one or more components and resolves them concurrently. Each lookup can still wait up to 7.5 minutes for its component's CI to publish the image, so a caller no longer waits for them one after another. Progress is streamed with the component name as a prefix, results are printed as component=image lines, and when any lookup fails every failed component is named in one error. forward-compatibility.yaml resolves its three images with one call. Signed-off-by: Abrar Shivani <ashivani@nvidia.com>
release-rc-assets.yaml installs Python, Go, PyYAML, regctl and operator-sdk, then validates the OLM bundle and builds and pushes it for amd64 and arm64. Move those steps into two composite actions so the daily main bundle can use them as well: - setup-olm-bundle-tools installs the tools. - validate-and-push-olm-bundle validates the release metadata (when a release tag is given) and the bundle, logs in to GHCR, builds and pushes the image and returns its reference. The RC workflow runs the same commands as before. The operator-sdk download now uses curl -f, so a missing release fails at the download instead of at the first operator-sdk call, the Docker login and Buildx setup now run right before the build, and the build prints the pushed image. Signed-off-by: Abrar Shivani <ashivani@nvidia.com>
abrarshivani
force-pushed
the
oci-chart-operands-main
branch
from
October 16, 2026 22:53
11afc6b to
dca1d9c
Compare
Add publish-main-chart.yaml, which runs daily and can be started by hand on the default branch only, so non-main code is never published as a main build. Only scheduled failures alert Slack. prepare-main-values checks that the operator image for the commit exists and runs prepare-main-values.sh, which resolves the newest main image of every operand that publishes per-commit images and prepares the chart values once: - generate-values-overrides.sh now takes VALUES_PATH=IMAGE pairs, so one image can fill several values, and splits each reference into repository, image and version, because driver-manager.fullimage joins the three fields itself. - apply-values-overrides.sh merges them into values.yaml. It rejects paths that are not image values in the chart and checks that every image renders unchanged into the ClusterPolicy. - The operator, validator and node status exporter use the operator image of the same commit. dcgm, dcgm-exporter, gds, gdrcopy, the driver and the DRA driver stay on their released versions: the first four have no per-commit builds, and the driver and DRA driver use different tag schemes. The prepared values are shared with two publish jobs, both using the version 0.0.0-main.<UTC timestamp>.g<operator sha>. The timestamp keeps each build unique because operand images change without new operator commits, and makes the newest one sort last. - The chart goes into the existing charts/gpu-operator package through a new main_chart_version input of publish-helm-oci-chart.yaml, so it keeps the access the e2e hosts rely on. The per-commit chart is built exactly as before. - The bundle CSV is updated with update-csv-images.py, which pins each image by digest, checked to contain every override image, and validated and pushed with the shared OLM actions. Its CSV name and version are unchanged, so a main bundle is installed on its own. When both are published, push-main-build-record.sh pushes a record of the chart, bundle and image digests to gpu-operator/main-builds under the same version, so a night where only one of them was published never counts as a build. forward-compatibility.yaml moves to the new generator arguments. Signed-off-by: Abrar Shivani <ashivani@nvidia.com>
Forward compatibility resolved three operand images itself, published a chart with released operands and copied a values file onto the test host, so the published chart never contained what was tested. It now finds the newest build record with get-latest-main-build.sh and runs e2e against that build's chart, with the operator version taken from its tag. The script fails if the newest record is older than 48 hours or the chart's digest no longer matches the record. The limit leaves a failed Monday publish, which the publish workflow reports itself, with Sunday's build to test. The workflow runs on Mondays at 04:17 UTC, two hours after the daily publish, and logs in to GHCR so reading records does not depend on the package's visibility. e2e-tests.yaml gets chart_reference and chart_version inputs. Given both, it skips publishing a chart from the current commit and installs the given one; the e2e jobs carry an explicit condition because the skipped publish would otherwise skip them too. Either input alone is rejected, and the jobs check that operator_version matches the chart's appVersion, since install-operator.sh sets operator.version. Callers that pass no chart publish and test exactly as before. The values-file input and its copy steps are removed. Signed-off-by: Abrar Shivani <ashivani@nvidia.com>
abrarshivani
force-pushed
the
oci-chart-operands-main
branch
from
October 16, 2026 23:16
dca1d9c to
101e1ce
Compare
rahulait
reviewed
Oct 16, 2026
| set -euo pipefail | ||
|
|
||
| readonly IMAGE_REFERENCE_EXPRESSION='eval(strenv(VALUES_PATH)) | (.repository // "") + "/" + (.image // "") + ":" + (.version // "")' | ||
|
|
Contributor
There was a problem hiding this comment.
Can we re-use .github/scripts/merge-yaml.py instead of this additional bash script to merge yamls?
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Publish a daily Helm chart and OLM bundle with main-branch operand images for CI and QA, and run forward compatibility against them.
publish-main-chart.yaml(daily) resolves the newest main image of each operand, prepares the chart values once, and publishes the chart, the bundle and a build record (chart, bundle and image digests), all tagged0.0.0-main.<timestamp>.g<sha>. The record is pushed only when both chart and bundle were published. DCGM, DCGM exporter, GDS, GDRCopy, the driver and the DRA driver stay on released versions.update-csv-images.py. Its name and version are unchanged, so a main bundle is installed on its own, not through a catalog.forward-compatibility.yaml(Mondays) installs the chart of the newest recorded build, and fails if it is older than 48 hours or its digest changed.e2e-tests.yamlcan now install a published chart viachart_referenceandchart_version.Using a main build
Checklist
make lint): no Go changes; shellcheck and actionlint are clean on the changed scripts and workflows apart from findings that already exist on mainmake validate-generated-assets): no generated assets touchedmake validate-modules): no module changesTesting
curl,regctl,helmandsleep.operator-sdk bundle validateandmake validate-csvpass.