Skip to content

Reconcile validation authority with immutable execution profiles (blocked foundation) - #206

Closed
justsml wants to merge 6 commits into
mainfrom
integration/validation-foundation-20260912
Closed

justsml wants to merge 6 commits into
mainfrom
integration/validation-foundation-20260912

Conversation

@justsml

@justsml justsml commented Sep 12, 2026

Copy link
Copy Markdown
Owner

Preserves the reviewed #158 integration without admitting it to main. The immutable profile and newer background/resume hardening are retained; authority provenance tests and typecheck pass.

Blocked: production tool/workflow call sites do not provide resolveExecutionAuthority, and the public tool does not pass the self-mode decision. Wire server-owned task scope, target authorization, policy decision and exact durable approval, then verify through the public tool seam. The four-mode paid matrix remains closed. This draft also contains the prerequisite #196/#152/#157 foundation commits; it must be refreshed after those merge. No paid model calls or human-review approvals were performed.

# Conflicts:
#	evals/validation-authority-foundation.ts
#	src/server/chat/security-research-runtime-context.ts
#	src/server/chat/securityResearchTurn.ts
#	src/server/research/execution-profile.ts
#	tests/integration/scheduler-backed-research.test.ts
@justsml
justsml changed the base branch from integration/main-20260912 to main September 13, 2026 19:48
@justsml

justsml commented Sep 13, 2026

Copy link
Copy Markdown
Owner Author

Retargeted from integration/main-20260912 to main. That old base is fully merged (0 commits ahead of main), which was inflating this PR's diff to 46 files / +4824; the true remaining delta is 19 files / +1148 −29. It now shows conflicts against main and needs a rebase.

Do not merge as-is. Verified against current main, the held reason in docs/branch-integration-2026-09-12.md still stands and is worse than the one-line summary suggests:

  1. The execution-authority resolver is never constructed in production — resolveExecutionAuthority exists only as a test stub (tests/integration/validation-authority-provenance.test.ts:120). All four executor sites in src/mastra/tools/validation-plan.ts (:393, :415, :437, :459) and src/mastra/workflows/validation-plan-workflow.ts:60 omit it, so resolvePayloadAuthorities returns an empty map and every payload denies server-authority-not-resolved.
  2. Self mode is unreachable — modelDecision is never supplied by the tool, so self mode denies every payload and can never confirm.
  3. Latent breaking change — requireModelValidationAuthority throws on plans with no pinned authority, so plans from src/mastra/tools/finding-helpers.ts:156, from the workflow, and any already persisted, would hard-fail record_result and confirm. This must land in the same change.
  4. Riskiest line: executor.ts:414-418 returns unconditional true for auto/self, relying entirely on the per-payload resolver to fail closed. A mis-wired or permissive resolver silently removes the destructive-payload guard. Needs a direct test before anything here merges.

Good news on the two fields that looked like they had no producer: policyDecisionId can anchor to ContainmentPolicySnapshot.id from resolveContainmentPolicy (merged via #152), and taskScopeVerified derives from the existing findTargetAuthorizationCoverage. Neither needs inventing.

Tracked by #100.

@justsml

justsml commented Sep 14, 2026

Copy link
Copy Markdown
Owner Author

Superseded by #216, which completes this work on a fresh rebase onto current main.

#216 builds the production resolver that was missing here (it existed only as a test stub), wires all four executing tool actions plus validation-plan-workflow, makes self mode reachable, adds the operator mode selection, and handles the unpinned-plan migration that would otherwise have hard-failed record_result/confirm on existing plans.

It also fixes a real bug the adversarial test exposed: a throwing resolver escaped executePlan and crashed the tool instead of denying — which, given that executionApprovalGranted returns unconditional true for auto/self, was the difference between failing closed and having no destructive-payload guard at all.

Closing this draft. Branch integration/validation-foundation-20260912 is preserved.

@justsml justsml closed this Sep 14, 2026
justsml added a commit that referenced this pull request Sep 16, 2026
)

* Rebase validation authority foundation onto main

Squash the PR #206 integration branch onto current origin/main (107
commits ahead) and reconcile the eval row schema: main's per-quartet
matched-row validation now reads `effectiveAuthorityMode`, and the
requested/effective parity check folds into the same pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Wire validation execution authority into production

The authority model shipped with no production resolver: the executor's
resolveExecutionAuthority was an optional dependency that only a test stub
ever supplied, so every non-yolo payload denied with
server-authority-not-resolved and no mode but yolo could execute.

Add src/server/validation-plans/authority-resolver.ts as the production
resolver. Per payload it resolves target authorization coverage from the
ledger, derives task scope from that authorization, resolves a containment
policy snapshot whose content-addressed id serves as both the policy
decision and the containment snapshot, creates the Tool Run, and settles
the durable approval: strict requires an exact operator approval covering
validationPayloadApprovalIntent(planId, payloadId), auto mints and consumes
its own exact one-time approval, self claims none. It never throws; every
missing fact becomes a denial record.

Wire it through both public seams. All four executing tool actions and the
validation plan workflow now construct the resolver, and both accept a
typed model decision (modelProceed / modelConfirm) so self is reachable
without widening model authority. executeMastraSecurityAction now stamps
the real selection into Tool Run metadata, and finding-helpers pins the
authority on the plans it creates.

Harden the executor so a throwing resolver denies the payload instead of
aborting the run - executionApprovalGranted returns true for auto and self
and relies entirely on this per-payload guard.

Operators select the mode as project.settings.validationAuthorityMode,
plumbed through the chat route into run metadata. Absence means strict; the
mode is never inferred from approvalMode.

Plans persisted before this contract carry no pin and would have hard-failed
record_result and confirm. They are adopted as self with a recorded
validationAuthorityMigration, preserving their original model-asserted
semantics without granting a stronger claim.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Apply Biome formatting and import ordering

`pnpm check` now exits 0. Fixes organizeImports in trajectory-export,
authority-resolver and the two validation-authority test files, plus
formatting in security-actions/execution.ts.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant