Repository navigation
feat(policy): revalidate pending policy proposals when effective policy changes #1636
Description
Activity
This issue has had no activity for 14 days and is now marked stale. It may be closed in 7 days if there is no further activity. Comment or remove the state:stale label to keep it open.
Stale validation seems like the right failure mode to make explicit here.
I would treat approval as invalid unless the reviewer is looking at a validation result tied to the same execution baseline that will receive the merge. The baseline probably needs a digest or revision tuple for:
- effective policy;
- provider and credential composition;
- approval settings that affect what is allowed;
- prover version or rule pack, if the verdict is not stable across versions.
Then each pending chunk can carry validation_status: current, stale, revalidating, failed, or superseded. The approve path should only accept current, or should atomically revalidate immediately before merge and record that newer verdict on the approval event.
Regression cases I would pin:
- proposal validated, effective policy changes, approve is blocked or revalidated before merge;
- provider or credential binding changes, old no-new-findings verdict cannot be reused;
- two pending chunks are approved after one changes the effective policy, the second is forced through the stale path;
- reviewer inbox shows stale state before approval, so a reviewer agent cannot reason from old findings;
- revalidation failure leaves a terminal approval outcome rather than silently falling through to merge.
That keeps the UX honest: pending approvals are not just queued text edits. They are queued decisions against a specific policy and provider state, and the system should fail closed when that state has moved.
This issue has had no activity for 14 days and is now marked stale. It may be closed in 7 days if there is no further activity. Comment or remove the state:stale label to keep it open.
Problem Statement
Pending policy proposal chunks store
validation_resultfrom the effective policy and credential/provider state at proposal submission time. If the effective policy, provider/credential composition, or relevant approval settings change before a pending chunk is approved, the stored prover verdict can become stale. Reviewers may seeprover: no new findingsor older findings that no longer match the policy that will actually receive the merge.Proposed Design
Track enough validation baseline context to identify stale pending chunks, such as the policy hash/revision and a provider or credential composition fingerprint used for validation. When those inputs change, re-evaluate pending chunks or mark their
validation_resultstale until revalidated. Approval paths should not silently rely on stale validation; they should refresh the verdict or surface a clear stale-validation state before approval.Alternatives Considered
Re-running the prover only at submit time is enough for the MVP and common single-proposal flow, but it does not cover long-lived pending chunks or concurrent policy/provider updates. Re-running only during human approval is smaller, but still leaves stale reviewer inbox state and does not help reviewer agents that reason before approval.
Agent Investigation
PR #1528 validates each proposal against a snapshot from
current_effective_policy_for_sandboxinsidehandle_submit_policy_analysis.validation_resultis persisted on the draft chunk.handle_approve_draft_chunkandhandle_approve_all_draft_chunksmerge pending chunks without recomputing the prover verdict against the latest effective policy and provider state.Related: #1528, #1062, #1434.