ci(tests): run the full suite on every main push

[seonghobae]

Current owner state — 2026-09-20 KST

This Draft is the canonical owner lane for the repository-wide/full-suite quality gate carried by trusted-uv-materializer-quality-ci.yml.

• protected base: main@e6334e229581a918e2f22de18733b76fa65d7e71
• exact current head: c965664a6d7fe0b75bf7ea019a9059220a32f06b
• open / Draft; current exact-head acceptance must be reacquired after head movement
• predecessor 169852c7fd0d0a451f1a3ef595b0922298ff331a ordinary-reconciled this owner with current protected main and reduced the protected-base production delta to the canonical Trusted-uv workflow

Consumer RED and dependency-closure repair

.github#2269@4dcd25c9f2789e4b8acbeef603e118dd80bfa014 exposed the repository-wide quality failure in run 35414335010 / job 105819872185: the focused materializer gate passed, then the complete repository suite failed during collection with ModuleNotFoundError: No module named 'defusedxml' because the workflow installed only the OpenCode lock while the Noema document surface requires the already hash-locked Noema dependency closure.

The retained production repair installs and caches both requirements-opencode-review-ci-hashes.txt and requirements-noema-document-ci-hashes.txt, and the PR trigger includes the Noema lock. No unpinned install, test exclusion, warning suppression, continue-on-error, or gate weakening is introduced.

Current-head contract reconciliation

Review 5259161939 found that protected-main reconciliation preserved the intended single hardened job + every main push workflow, but the retained contract test had reverted to the older two-job/path-filter assumptions. That stale test required trigger paths twice, two harden/checkout blocks, two PR-head-only refs, and the old one-line dependency install, while also losing the Noema-lock trigger assertion.

Commit c965664a6d7fe0b75bf7ea019a9059220a32f06b repairs the test contract rather than weakening the production workflow. It now binds:

• PR path triggers once, including the Noema dependency lock;
• unfiltered push: main so the full suite runs after every accepted main movement;
• one hardened checkout with github.event.pull_request.head.sha || github.sha exact-head semantics;
• pinned actions, read-only contents permission, Ubuntu 24.04 and bounded timeout;
• both hash-locked dependency closures in cache/install;
• retained focused tests, complete branch coverage, docstring and compile gates.

No predecessor hosted evidence transfers to c965664a....

Fresh exact-head evidence

The natural current-head generation is terminal but not accepted:

• Python Security 35484950365: CANCELLED
• SAST Semgrep 35484950385: CANCELLED
• Security Scan 35484950398: CANCELLED
• Trusted uv Materializer Quality CI 35484950390: CANCELLED
• CodeQL PR 35484950369: CANCELLED

This is missing hosted acceptance, not source GREEN and not proof of a product/test failure. The cancellation/runner-admission RCA remains with .github#712; do not manufacture another generation with a source-neutral wake commit or blind unchanged-head rerun.

Ownership boundary

#1911 owns the repository-wide/full-suite Trusted-uv quality workflow and dependency-closure contract. .github#2040 may retain distinct CodeQL scheduler/credential/runtime-quality work but must consume this owner losslessly rather than keep a parallel Trusted-uv implementation. #1911 does not absorb #2269 authenticated redirect containment, #2291 Strix trusted-binder runtime, or #2272 Pages shell-input security.

Keep this PR Draft until exact c965664a... has terminal-success hosted evidence and the live protected-branch review requirements are satisfied. No force-push, destructive rebase, self-approval, administrator bypass, synthetic status, source-neutral wake commit, unchanged-head blind rerun, duplicate owner, predecessor receipt transfer, or gate weakening is authorized.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[seonghobae]

Failing checks on 37312475 classified; recovery = one push merging current main (as c36533be), no code change.

• noema-review (run 33971548451, created 2026-09-05T14:21Z — the old-core straggler push, sidecar on the pre-efb89269 pin, no per-route artifact yet): route preflight passed after 393 s, then the gateway walked for 2704.7 s and answered 502 Bad Gateway (served_model = the last candidate tried, deepseek-v4-flash). That is failure class (1), the orchestrated no-tools walk ending in 502 — see the census on Scheduler pre-review update-branch discards in-flight checks under queue saturation: 76/77 merges since 09-04 had 0/12 required contexts at merge time #1935. Nothing in this PR's single new workflow file is on that path.
• CodeQL compatibility analysis (actions|python) and opencode-review: the designed pending-verdict states while OPENCODE_REPOSITORY_DISPATCH_ACTOR still names the legacy dispatcher (CodeQL dispatch terminal status publication remains unproven across repositories #1929, owner-gated); the compatibility shards intentionally fail after dispatching, and the dispatched scan's verdict cannot be delivered until the variable changes.
• The CodeRabbit comment is its "no automatic review under 10 stars" notice — nothing to address.

The branch was 22 commits behind main; a re-run would re-execute the sidecar bound at run creation, so the head is refreshed instead with a merge of main@fb2ae81d. The merged tree was gated locally (GITHUB_ACTIONS=true): 2920 passed, 1 skipped, 21 subtests passed in 140.69s (0:02:20);TOTAL 13181 0 5326 0 100%;RESULT: PASSED (minimum: 100.0%, actual: 100.0%). Side effect worth having: the new head's Noema/Strix runs are among the first created after fb2ae81d, so their sidecar artifacts carry deferred_count (#1947), candidate/probed/skipped/ready_count (#1949) and unexpected_exception type= (#1950) together.

[seonghobae]

CodeQL compatibility analysis (actions) on c36533be — same designed state as before, no push. The shard's real output (after ##[endgroup]) is CodeQL scan dispatched. The dispatch workflow will rerun this exact failed CodeQL job after publishing its terminal verdict. — the compatibility shard fails intentionally after dispatching. Its dispatch runs for this head are queued (34021293034 at 08:14:20Z, 34021357120 at 08:15:40Z, one per language) and were sent with the app token, so they will hit the identity gate the same way every dispatch did today (a peer's live test at 07:02Z, right after the variable was touched at 07:01:34Z, still rejected actor=sender=opencode-agent[bot]; see #1929). Nothing in this PR is on that path; the verdict arrives when the owner-gated allowlist admits the dispatcher, and the dispatch workflow then reruns this exact job.

[seonghobae]

noema-review on c36533be (run 34017954595, job 08:19:19–08:21:43Z, artifact noema-sidecar-evidence 9985677359) — class (4) preflight-fail (capacity); no push. This is the first Noema artifact for this PR on the post-fb2ae81d launcher and it carries the new fields: candidate_count 24 / probed_count 6 / skipped_count 18 / ready_count 0 / deferred_count 0 / rejected_count 6 (target_ready 8, probe_budget 16, account_skip_after_429 2). All three credential accounts answered 429 to their first two probes (nvidia_nim ×2, nvidia_nim_sub ×2, openrouter ×2), the account-skip rule dropped the remaining 18 candidates without probing, and with nothing ready the sidecar failed closed before healthz — the provisioning step failed 2 min 24 s in. No serving phase ran, so no unexpected_exception line, as expected for the no-tools shape.

Same shape as the other post-#1949 samples in this hour (contextual-orchestrator#1078's strix run, the main-push strix run): the launcher's levers behave as designed and the pool is rate-limited end to end — the capacity item on #1948. A push cannot change an upstream 429; the branch already carries current main.

[seonghobae]

현재 c36533beb63ba980c2c520fc9fe5c4c448937c8b와 main43024633 기준에서 중복을 확인했습니다. 새 main-full-suite-gate와 기존 trusted-uv-materializer-quality-ci의 full-quality-gate가 materializer main 변경에 동일한 전체 pytest/coverage를 각각 실행합니다. 기존 파일의 3.10 compile/import와 3.14 전체 품질 검사를 한 runner로 순차 통합하고, main push만 경로 필터를 없애며 전체 docstring 검증을 승계하는 최소 후보를 격리 worktree에서 준비합니다. PR 경로 필터, 두 Python 버전, 읽기 전용 권한과 SHA pin, PR/head 및 push/SHA 구분은 유지합니다. 삭제되는 별도 3.10 check 이름은 현재 main 보호 규칙의 required contexts에 없음을 직접 확인했으며 보호 설정은 바꾸지 않습니다. 새 workflow의 기능을 완전히 승계한 뒤 해당69줄 파일만 제거하고 Git 이력은 보존합니다. 기존 PR/원격 head를 일반 갱신 직전에 재조회하며 활성 작성자가 있으면 알려 주세요. #1879 응답 정리 수정의 보호 병합 전에는 경고 엄격 전체 검사 결과를 별도 prerequisite로 구분합니다.

[seonghobae]

opencode-review on c36533be is the receipt gate, not a sidecar failure — and the head has since moved; no push from me.

• The opencode-review check fails with No APPROVED or CHANGES_REQUESTED from opencode-agent on the current head: the required workflow only verifies a receipt. The review itself runs in opencode-review-dispatch.yml, whose validate-pr-metadata rejected 44 of today's 50 failed runs with actor=sender=opencode-agent[bot] — the owner-gated allowlist (CodeQL dispatch terminal status publication remains unproven across repositories #1929). A live test right after the variable was touched at 07:01:34Z still rejected, so the gate value has not changed yet. Nothing in this PR is on that path.
• Head moved from c36533be to ad245962 at 09:21:48Z (another session merged current main, which now carries fix(strix): name the sandbox bootstrap failure and give it a bounded retry #1953, and added a doctoring document; c36533be is an ancestor, nothing lost). That cancelled the Strix scan that had been running on c36533be since 07:00Z — the cost of moving a head while a scan is in flight. The new head binds the fix(strix): name the sandbox bootstrap failure and give it a bounded retry #1953 sidecar, so its Strix run is the first sample that can show the STRIX_SANDBOX_UNAVAILABLE verdict token if the sandbox class recurs.
• Earlier on this head: noema-review was class (4) preflight-fail (all three accounts 429 twice, ready 0, fail-closed in 2 min 24 s — artifact 9985677359), and both CodeQL compatibility shards are the designed post-dispatch failure.

[seonghobae]

Python 3.14 full quality gate (run 34024435377, step 11) — 11 failures, one signature, and it is not this pull request's content. Every failure carries the same line:

pytest.PytestUnraisableExceptionWarning: Exception ignored while calling deallocator
<function _TemporaryFileCloser.__del__ at 0x…>: None

What the evidence rules out:

• Not this branch's diff. The head adds main-full-suite-gate.yml and a doctoring document; the 11 failing tests are in test_noema_review_gate.py, test_noema_repair_attempt_telemetry.py, test_pingora_edge_policy.py, test_repository_metadata_live_verification.py and test_sandboxed_web_e2e.py, none of which this pull request touches, and their own code builds urllib error bodies from io.BytesIO, not from temporary files.
• Not the whole tree today. The same workflow passed at 08:23:22Z (ad48dd65) and 08:38:53Z (4cf6febc) on two other branches.
• Not the previous failures of this workflow. main's last two failures (09-04) were 17 scheduler tests with no unraisable signature, and the 01:29Z failure on another branch was a single test, also without it.

The signature itself is a finalizer running during garbage collection, which pytest attributes to whichever test happens to be executing at that moment — so the failing test names do not locate the leaked object, and a deterministic reproduction needs a Python 3.14 interpreter (the toolchain available to me here is 3.12). I am not pushing a speculative change on that basis.

The cheap discriminator is a re-run of this one workflow: it runs no review sidecar, so re-running does not resurrect stale trusted code the way re-running a review job would. Triggered now. If it passes, this was finalizer timing; if it fails identically, the leak is real and I will bisect it against the recent main commits this head is the first to carry.

[seonghobae]

Root cause found (a peer, from local git only; I verified it the same way): this branch's own -W error promotes a pre-existing warning into 11 errors.

ad245962   python -m coverage run -m pytest tests -q -W error
main       python -m coverage run -m pytest tests -q

That one flag turns PytestUnraisableExceptionWarning — raised when _TemporaryFileCloser.__del__ runs during garbage collection — into a failure, and pytest attributes it to whichever test happens to be executing at that moment. Every observation now fits: one signature across 11 unrelated tests, other branches passing the same gate today (they have no -W error), the failing test names not locating the leak, and my 3.12 run not reproducing it (I ran without the flag).

So this is not a regression in main and not a defect in the tests that failed. The leaked closer is not this repository's code either: .github contains exactly one NamedTemporaryFile (scripts/ci/install_strix_timeout_compat.py:78, inside a with), plus 19 TemporaryDirectory and 2 mkstemp; the finalizer almost certainly belongs to a dependency.

The design decision belongs to this pull request, because -W error and its guard test test_full_suite_command_rejects_warnings are what it adds. Three ways to keep the gate green:

1. -W error -W default::pytest.PytestUnraisableExceptionWarning — keeps every other warning fatal and demotes only the third-party finalizer noise.
2. -p no:unraisableexception — turns the plugin off, which also hides unraisable exceptions that would be ours.
3. -W error::UserWarning — matches exactly what the new test asserts (it emits a UserWarning fixture), but stops rejecting every other warning class.

I recommend (1): the guard test still fails on its UserWarning fixture, the stated intent ("reject warnings") survives for everything the repository controls, and the exclusion is narrow and self-documenting. I have re-run the failed job to see whether the promotion is deterministic or GC-timing dependent, and will push (1) unless the branch's author prefers another option — say so here and I will not touch it.

[seonghobae]

Pushed 9e846138: option (1), with the control run first. The gate command is now

python -m coverage run -m pytest tests -q -W error -W default::pytest.PytestUnraisableExceptionWarning

I reproduced the mechanism myself on Python 3.12 before changing anything, using a fixture whose __del__ raises (CPython cannot propagate that, so it is always unraisable, and pytest's plugin converts it):

arm	command	result
1	no flags	1 passed, 1 warning — the stimulus is real
2	-W error	1 failed — the escalation, reproduced
3	this branch's new command	1 passed, 1 warning — the exemption works
4	a UserWarning under arm 3's flags	1 failed — the branch's own contract survives

Arm 1 matters as much as arm 2: without it a negative result would have meant "no stimulus", not "no escalation". Arm 4 is why this is option (1) and not -W error::UserWarning — the latter also makes the suite green while quietly reducing "reject warnings" to one class, and test_full_suite_command_rejects_warnings would still pass either way. A peer ran an independent three-arm version and got the same result. Because it reproduces on 3.12, this is not Python 3.14 behaviour; the flag is the cause.

The seven guard tests in test_trusted_uv_materializer_quality_workflow_contract.py pass on the new head, and the exemption is commented in place with the run id so the next reader does not have to rediscover why one class is exempt. If the branch author wants the flag left alone and the dependency finalizer chased instead, revert this commit and say so — the evidence above stands either way.

[seonghobae]

Correction to the comment above: option (3) does not work at all, so "both options make the suite green" was wrong. A peer caught it and I confirmed it here:

PytestUnraisableExceptionWarning -> PytestWarning -> UserWarning -> Warning
issubclass(pytest.PytestUnraisableExceptionWarning, UserWarning) is True

So -W error::UserWarning catches the unraisable warning as well. Against the same __del__-raises stimulus: -W error::UserWarning → 1 failed; the committed -W error -W default::pytest.PytestUnraisableExceptionWarning → 1 passed, 1 warning. Option (3) leaves the gate red.

That also changes what arm 4 proves. It is not "which of two working options to prefer" — it is the independent evidence that option (1) keeps the branch's own contract: with the blanket -W error still in place, a plain UserWarning is still fatal, which is exactly what test_full_suite_command_rejects_warnings was written to guarantee.

The pushed commit 9e846138 is unaffected; only my description of the alternative was wrong. Recording why it mattered: I rejected option (3) on an intent argument ("it narrows what the author bought") and never ran it, and the peer who proposed it never ran it either. A rejected option that was never executed comes back the moment someone re-opens the design argument — and this one would have turned the gate red again. Running an option you intend to reject costs a second.

[seonghobae]

Conflict resolved and pushed as 21e128bf (merge of main@5ea1cc47, parents 9e846138 + 5ea1cc47; no rebase, no force-push beyond the lease on the previous head).

One conflict, CHANGELOG.md, and it is the prepend-sibling shape: this branch adds a section at the top and #1958 added one at the same place. Both entries are kept — this branch's "Main quality checks share one runner" stays on top as the not-yet-merged change, with #1958's dispatch-concurrency entry directly below it, which is the order the file will have once this merges. No code conflicts.

Checks for the touched file: both test files that read CHANGELOG.md pass (39 passed), and the workflow contract for the file this branch actually changes passes (7 passed). Conflict markers: zero.

Running note for whoever picks this up: the branch is a draft, so nothing here flips it.

[seonghobae]

Second conflict resolved and pushed as 8761c004 (merge of main@0b0f1047; both changelog entries kept, this branch's on top as the not-yet-merged change, #1957's below it). Same shape as the last one: CHANGELOG.md only, no code conflicts, zero markers. Checks for the touched files: the two test files that read the changelog plus this branch's own workflow contract — 46 passed.

This is the third prepend collision on this branch today (#1953, #1958, #1957 each landed a section at the top while this one sat open). If it keeps costing round trips, the durable fix is to give the file a stable insertion point — a marker line the tooling inserts under — rather than having every branch write to line 1. Not something to change inside this pull request.

[seonghobae]

Fourth resolution today, pushed as 664a5dd9 (merge of main@c232ca03; CHANGELOG.md only, no code conflicts, zero markers, 24 sections with this branch's entry on top). Checks for the touched files: 46 passed.

The four collisions on this one branch, each caused by a different merge landing a section at line 1 while this pull request sat open:

resolved as	invalidated by
21e128bf	#1953
(same push window)	#1958
8761c004	#1957
664a5dd9	#1959

Across the repository the pattern is measurable: 21 of the last 29 commits on main touched this file, and three merges landed within 27 minutes this evening. Every one of these resolutions took the same shape — one file, both entries kept, no code involved — which is the signature of a serialization point rather than a disagreement. A separate pull request is addressing the file's insertion strategy; I am not duplicating that work here, and this branch stays a draft.

[seonghobae]

Fifth resolution, pushed as 2f0729d9 (merge of main@dd0b96fe; CHANGELOG.md only, no code conflicts, zero markers, 25 sections with this branch's entry on top). Checks for the touched files: 46 passed.

Five collisions on this branch today, one per merge that landed while it stayed open: #1953, #1958, #1957, #1959, #1960. Each was the same shape — one file, both entries kept, no code involved.

[seonghobae]

Fresh Naruon consumer read confirms #1911 is still the canonical repository-wide Trusted uv/full-suite dependency-closure owner, but its current exact dfeadc7adfb02d73e166104f2987f14fbf7fe82e now diverges from protected main@e6334e229581a918e2f22de18733b76fa65d7e71 by 22 ahead / 33 behind (merge base 64aa08d7...).

Meanwhile #2040 advanced from 652764a... through RED 82a4e2a... and GREEN 7901ca5... with the same Noema lock trigger/cache/install invariant already owned here. Treat that as consumer evidence / duplicate implementation, not a new owner transfer.

Minimum owner-path next step: ordinary/non-force reconcile #1911 onto current protected main, retaining the full four-file owner contract and auditing shared workflow/test changes path-wise. Then prove whether #1911 completely carries #2040's two new dependency-closure commits. If so, downstream #2040 should adopt/adapt this canonical owner rather than retain a parallel Trusted-uv implementation. Do not force-push, close either PR merely because of overlap, or transfer predecessor Checks/reviews across the reconciled head.

[seonghobae]

Fresh Naruon consumer audit found one owner-topology drift that should be repaired before this lane can be accepted. Protected central main is now e6334e229581a918e2f22de18733b76fa65d7e71, while this PR's exact head dfeadc7adfb02d73e166104f2987f14fbf7fe82e is still based on the earlier protected tip 64aa08d7fa487deacd41c761c36277ca68cab6c9.

Please ordinary/non-force adopt the current protected main again, preserving this lane's bounded four-file Trusted-uv/full-suite contract and dropping any stale collateral exactly as the earlier reconciliation did. Do not force-rebase or transplant the old tree wholesale. Re-census the effective diff after adoption and require it to remain bounded to the canonical owner files.

This also matters because #2040 currently carries an overlapping Trusted-uv dependency-closure repair at 7901ca565fb93a9e869cd4136476384ac3a07bdf. Keep #1911 as the canonical full-suite/Trusted-uv owner; #2040 should converge through this owner rather than establish a parallel owner for the same workflow. No predecessor checks/reviews transfer after the protected-main adoption.

[seonghobae]

Current-head finding: protected-main reconciliation preserved the canonical single-job / every-main-push workflow, but tests/test_trusted_uv_materializer_quality_workflow_contract.py reverted to the older two-job/path-filter assumptions. The test currently requires each trigger path twice, two harden/checkout blocks, two exact PR-head refs, and the old one-line quality-tool install, while the production workflow intentionally has unfiltered push: main, one hardened/checkout job, github.sha fallback for push, and a folded two-lock install including requirements-noema-document-ci-hashes.txt. This also drops the test-first assertion that the Noema lock itself triggers PR evidence. Treat the stale contract as the repair target, not the current canonical workflow: update the test to bind the single-job/every-main-push design, Noema lock trigger/cache/install closure, pinned actions, and current exact-head checkout semantics. No hosted GREEN or predecessor evidence transfers after the repair head moves.

[seonghobae]

Fresh exact-head acceptance update (2026-09-20 KST): current head remains c965664a6d7fe0b75bf7ea019a9059220a32f06b, but the five pull-request-triggered runs previously described as queued are now terminal CANCELLED: Python Security 35484950365, SAST Semgrep 35484950385, Security Scan 35484950398, Trusted uv Materializer Quality CI 35484950390, and CodeQL PR 35484950369. Combined commit statuses currently show only successful Devin Review and CodeRabbit; no hosted required-family GREEN may be inferred. This is execution/control-plane evidence, not a source finding, so do not create a source-neutral wake commit or transfer predecessor receipts. Keep Draft; route queue/cancellation RCA through #712 and reacquire exact-head hosted evidence only through legitimate scheduler/capacity recovery.