Skip to content

fix(pipeline): stop skipping the NuGet package tester modules - #7127

Merged
thomhurst merged 1 commit into
mainfrom
fix/nuget-tester-modules-skipped
Oct 18, 2026
Merged

thomhurst merged 1 commit into
mainfrom
fix/nuget-tester-modules-skipped

Conversation

@thomhurst

@thomhurst thomhurst commented Oct 18, 2026 •

Copy link
Copy Markdown
Owner

Problem

Since #5752 (2026-04-26), TestNugetPackageModule, TestFSharpNugetPackageModule, TestVBNugetPackageModule and TestAspNetCoreNugetPackageModule have run zero tests on every OS while reporting Passed. CI logs show Skipping net10.0: no build output found for this test project. for every framework, and the modules finish in 7–160 ms.

TestBaseModule checks for bin/Release/<tfm> output before each run. These four modules don't use --no-build; they let dotnet run build the tester against the locally packed packages. Nothing builds them beforehand, so the check always fails.

Fix

Only run the output check when the module uses NoBuild = true. All other TestBaseModule subclasses set it, so their behaviour is unchanged.

These tests haven't run for about six months, so this PR's CI is the first real check of the packed packages through them.

Summary by CodeRabbit

  • Bug Fixes
    • Framework tests that build as part of execution are no longer skipped because prebuilt output is unavailable.

TestBaseModule probed bin/Release/<tfm> before every run so it could skip
TFMs a test project didn't build (#5752). The NuGet tester modules don't use
--no-build: they let dotnet run build against the packed packages, so the probe
never found any output and skipped every framework while reporting success.
Only probe runs that use --no-build.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-18T15:52:16.759301Z 7c0f428 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai

coderabbitai Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 072f228e-6351-4ace-9a01-4cb34fadc7ad
📥 Commits

Reviewing files that changed from the base of the PR and between c2f4b9a and 7c0f428.

📒 Files selected for processing: 1
  • tools/TUnit.Pipeline/Modules/Abstract/TestBaseModule.cs

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 5 remain after this review.

📝 Walkthrough

When NoBuild is true, ExecuteAsync checks for missing framework output. When building is enabled, it no longer skips execution because prebuilt output is missing.

Changes: Test Execution

Layer / File(s) Summary
Gate the framework output check
tools/TUnit.Pipeline/Modules/Abstract/TestBaseModule.cs
ExecuteAsync checks for missing framework output only when NoBuild is true. Runs that may build during execution proceed without this pre-check.

Priority: ⬇️ Low

Estimated code review effort: 1 (Trivial) | ~5 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to 7c0f4

The change lets NuGet tester modules build against locally packed packages instead of skipping when prebuilt output is absent. The other listed modules retain the existing output check, and no actionable merge risk remains.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 1 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the main change: preventing the pipeline from skipping NuGet package tester modules.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
  • Fix all pre-merge checks with AI
✨ Finishing Touches: 💡 1
📝 Generate docstrings: 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

A rabbit checks the build output with care,
NoBuild decides when to look there.
If building can happen along the way,
The tests proceed and start their day.
The rabbit hops on, pleased with the change.

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

Review

The fix is correct and minimal. HasFrameworkOutput only makes sense for --no-build runs, and the four NuGet tester modules (which let dotnet run build against the packed packages) were always skipped because nothing prebuilds them. Gating the probe on testOptions.NoBuild == true restores their execution, and the other modules I checked set NoBuild = true, so their behavior is unchanged.

Notes (non-blocking):

  • These modules have not run for ~6 months, so CI failures from this PR may be pre-existing breakage in the packed-package tests rather than regressions from the change. Triage accordingly.
  • Design suggestion: a silent 'Skipping ... reporting Passed' with zero tests ran is what hid this for 6 months. Consider failing the module (or logging a warning) when all frameworks are skipped, so this class of bug can't recur quietly.
  • The NoBuild == true check is implicit coupling between the skip logic and the build mode; a small comment (already added) is adequate.

LGTM.

@greptile-apps

greptile-apps Bot commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge and fixes the NuGet testers being skipped.

Summary

Updates TestBaseModule so only runs with NoBuild = true require existing build output.

  • NuGet package tests can build and run for each framework.

Reviews (1) · Last reviewed commit: "fix(pipeline): stop skipping the NuGet p..." · Reviewed by Greptile

@thomhurst
thomhurst merged commit 8af2dd3 into main Oct 18, 2026
24 checks passed
@thomhurst
thomhurst deleted the fix/nuget-tester-modules-skipped branch October 18, 2026 15:55
This was referenced Nov 1, 2026

This branch was previously deployed

1 inactive deployment
Pull Requests — 7c0f4287 Deployed Oct 18, 2026 by thomhurst via modularpipeline (ubuntu-latest) #20147
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant