Skip to content

validate-task.yml runs pytest --cov-report=xml with no --cov, so Python coverage is never collected #1199

Description

@ptr727

.github/workflows/validate-task.yml runs pytest --cov-report=xml with no --cov, which collects nothing and writes no coverage.xml. The upload step that follows is best-effort by design, so it stays green on the missing file, and Codecov receives nothing. That is exactly the failure WORKFLOW.md D1.6's own Prevents clause names, "coverage silently going unreported".

Found by the canonical-content pass on PR #1198 and raised independently by Copilot on the same pull request. Deferred out of that change, whose category is contract wording, because this one changes CI behavior for every Python repo in the fleet.

The invocation

.github/workflows/validate-task.yml, the unit-test job's Python step:

uv run pytest --cov-report=xml

pytest-cov writes a report only for what --cov selects. With --cov-report=xml alone and no --cov, no measurement runs and no file is written. Verified directly: uvx --with pytest-cov pytest --cov-report=xml produces no coverage.xml, and adding --cov produces "Coverage XML written to file coverage.xml".

The upload step reads files: ./coverage.xml under continue-on-error: true with fail_ci_if_error: false, which D1.6 requires so a Codecov outage never reds the gate. Those two settings also mean a permanently missing file is indistinguishable from a healthy run.

Why nothing catches it

The .NET half of D1.6 has a version floor and an invocation shape precisely because a silent-green coverage run was already found there once. The Python half has neither, and no gate reads whether a report was produced. A repo can be audited operational against D1.6 with coverage never reaching Codecov.

What the fix touches

  • The invocation in validate-task.yml, which needs --cov pointed at what each repo ships. The two profiles differ, so the selector has to work for a src layout and for a scripts tree.
  • WORKFLOW.md D1.6, which documents the broken form as the contract.
  • spec/project-types.json's python.coverage.codecov assert, which repeats the same string, so an audit currently certifies the defect.

Worth confirming against a real Python fleet repo's run rather than the hub, since the hub is source-only and its own unit-test job takes the skip path.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions