Skip to content

Test installed SOS artifacts in official builds - #6205

Merged
max-charlamb merged 15 commits into
mainfrom
feature/nightly-sos-package-tests
Nov 12, 2026
Merged

max-charlamb merged 15 commits into
mainfrom
feature/nightly-sos-package-tests

Conversation

@max-charlamb

@max-charlamb max-charlamb commented Nov 10, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Run installed SOS tests in official nightly and release builds instead of the normal test jobs.
  • Repackage the same build's prebuilt dotnet-sos and dotnet-dump outputs without rebuilding, install them in isolation on native Helix workers, and run the existing SOS suite on eight platforms.
  • Public/PR CI is unchanged, buildOnly still skips all tests, and installed tests do not gate packaging or publishing.

Validation

Official eight-platform validation run 3133729 at 63f7bb9c4986afa0aae3c2cd1f95f7b22f7449db (completed: all eight Core platform matrices passed; the known Windows Framework XML reporting limitation remains).

Max Charlamb added 3 commits November 10, 2026 12:09
Reuse packaged SOS test layouts from #6170 and add an exact-version nightlyPackageVersion mode to the existing diagnostics pipelines. Keep normal validation and publication unchanged; nightly mode installs tools from the public feed and runs isolated packaged tests without publishing products.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: dc2f081b-c77b-449b-a16c-7e09bf1cd9e2
Pass the caller-selected public/internal job template into the extracted nightly stage instead of resolving a root variable in the child template context.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: dc2f081b-c77b-449b-a16c-7e09bf1cd9e2
Move feed-backed SOS validation to its own manual pipeline and restore diagnostics.yml unchanged. Document the existing Framework empty-theory reporting limitation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: dc2f081b-c77b-449b-a16c-7e09bf1cd9e2
@max-charlamb
max-charlamb requested a review from a team as a code owner November 10, 2026 18:05
Run packaged SOS tests after the full build in parallel with packaging and publishing. Consume prebuilt same-run Windows x64 outputs, preserve build-ID versioning, and create unsigned local-only packages without rebuilding. Remove the standalone pipeline and retain optional local nightly-feed testing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: dc2f081b-c77b-449b-a16c-7e09bf1cd9e2
@max-charlamb max-charlamb changed the title Add feed-backed nightly SOS package validation Test installed SOS artifacts in official builds Nov 10, 2026
Skip normal test-only and Helix jobs for official nightly and release builds. Keep installed-tool tests enabled even with buildOnly, while preserving normal non-official test selection.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: dc2f081b-c77b-449b-a16c-7e09bf1cd9e2
Max Charlamb and others added 3 commits November 10, 2026 16:03
Keep installed-tool tests as the only official test suite, but skip them when buildOnly is enabled. Restore the buildOnly label and trim the SOS README addition.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: dc2f081b-c77b-449b-a16c-7e09bf1cd9e2
Helix rejected the internal installed-tool submission before creating a job because Creator was unset. Pass Build.RequestedFor as the internal job creator, preserving the existing public bot identity.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: dc2f081b-c77b-449b-a16c-7e09bf1cd9e2
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: dc2f081b-c77b-449b-a16c-7e09bf1cd9e2
Max Charlamb and others added 7 commits November 11, 2026 01:12
Use absolute source paths for Arcade's Unix build wrapper and enable packing binlogs so early failures still produce logs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: dc2f081b-c77b-449b-a16c-7e09bf1cd9e2
Skip explicitly Core-only theories before Framework shard discovery and validate xUnit XML in both Helix launchers so reporter errors cannot silently pass. Remove obsolete local setup scripts now superseded by the shared worker bootstrap and document the eight-platform release matrix.

Verified all five Windows x64 Helix shards: 3,479 passed, 55 expected skips, zero failures or XML errors. Official release template previews include all eight installed-test jobs and preserve buildOnly and independent shipping dependencies.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: dc2f081b-c77b-449b-a16c-7e09bf1cd9e2
Remove the coreOnly theory changes and Python XML validators. Defer the existing Framework discovery and reporting limitations to separate work.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: dc2f081b-c77b-449b-a16c-7e09bf1cd9e2
Keep the existing Helix template focused on Bin tests, inline the installed-test stage, and use a dedicated packaged-test job template. Simplify test-only helpers and verify dbgeng and SOS load from the selected layout across all runtime versions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: dc2f081b-c77b-449b-a16c-7e09bf1cd9e2
Store prefixed EngineHost startup messages on the target so tests can validate the actual dbgeng and SOS module paths without a duplicate protocol command.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: dc2f081b-c77b-449b-a16c-7e09bf1cd9e2
Move cDAC preparation to ToolLayoutExtensions, keep Bin layout filename resolution self-contained, and remove low-value harness implementation tests per review feedback.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: dc2f081b-c77b-449b-a16c-7e09bf1cd9e2
Include the latest .NET 11 servicing dependency update before rerunning the installed-tool validation matrix.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: dc2f081b-c77b-449b-a16c-7e09bf1cd9e2
@max-charlamb
max-charlamb enabled auto-merge (squash) November 12, 2026 04:53
@max-charlamb
max-charlamb merged commit 3ca10b4 into main Nov 12, 2026
25 checks passed
hoyosjs added a commit that referenced this pull request Nov 12, 2026
Snap main at 3ca10b4 into release/stable, following #6216.

Includes #6212, #6220, #6221, #6224, #6218, and #6205. Preserves stable
package versioning: the only content difference from that main snapshot
is the empty PreReleaseVersionLabel.

Dependency metadata conflicts were resolved using the exact main files.
cDAC packaging remains enabled.

---------

Co-authored-by: Juan Hoyos <19413848+hoyosjs@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: dotnet-maestro[bot] <42748379+dotnet-maestro[bot]@users.noreply.github.com>
Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
Co-authored-by: Max Charlamb <maxcharlamb@microsoft.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: David Wrighton <davidwr@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants