Skip to content

#100: Make validation execution authority explicit and fail closed - #216

Merged
justsml merged 3 commits into
mainfrom
work/validation-authority-100
Sep 16, 2026
Merged

justsml merged 3 commits into
mainfrom
work/validation-authority-100

Conversation

@justsml

@justsml justsml commented Sep 14, 2026

Copy link
Copy Markdown
Owner

Closes #100. Unblocks #123, #203, #67, #68 and the #60–#66 qualification cluster. Last open blocker on umbrella #97 alongside the custody chain.

Supersedes draft PR #206, which carried the authority model but never connected it.

The problem

The branch defined a complete authority model — ValidationAuthorityMode (strict | auto | self | yolo) pinned immutably per plan, and ValidationExecutionAuthority as the server-owned fact record — but the resolver was never constructed in production. It existed only as a stub in one test. With no resolver and mode ≠ yolo, resolvePayloadAuthorities returned an empty map and every payload denied with server-authority-not-resolved. Self mode was unreachable because modelDecision was never supplied by the tool. Nothing anywhere set metadata.validationAuthorityMode, so the effective mode was always strict in production.

What this adds

src/server/validation-plans/authority-resolver.ts — the production resolver. Per payload it gathers target authorization from the ledger (findTargetAuthorizationCoverage, falling back to the plan target's locator for manual and evidence-only payloads), derives taskScopeVerified from that authorization's scope, takes a resolveContainmentPolicy snapshot whose content-addressed id serves as both policyDecisionId and containmentPolicySnapshotId, creates a real Tool Run, then resolves the durable approval:

  • strict requires an exact operator approval covering the exported validationPayloadApprovalIntent(planId, payloadId)
  • auto mints and consumes its own exact one-time approval
  • self claims none

It never throws — every missing fact becomes a denial record.

Note the two fields that appeared to have no producer do in fact have one: policyDecisionId/containmentPolicySnapshotId anchor to the containment snapshot from #152, and taskScopeVerified derives from the existing authorization ledger. No new identifier scheme was invented.

Both public seams wired. All four executing tool actions and validation-plan-workflow now construct the resolver; the workflow input schema gained validationAuthorityMode, threadId and durableApprovalId, having previously had no authority field at all. executeMastraSecurityAction passes the real selection into Tool Run metadata — previously every Tool Run recorded default strict regardless of actual mode. finding-helpers pins authority on plans it creates.

Self mode reachable. Flat modelProceed/modelConfirm booleans on the tool and workflow input schemas — no unions or combinators, per the schema-subset rule — threaded into every executePlan call.

Operator selection. project.settings.validationAuthorityMode, modelled as a meaningful value where absence means strict, never inferred from approvalMode. Validated in projectAdapter and projects/setup, plumbed through the chat route into run metadata and onward to the request context the tool reads.

A real bug the tests found

executionApprovalGranted returns unconditional true for auto and self, relying entirely on the per-payload resolver to fail closed. Writing the adversarial test exposed that a throwing resolver escaped executePlan and crashed the tool rather than denying. It now becomes a per-payload denial. Given what that line delegates, an exception escaping the resolver was the difference between fail-closed and no destructive guard at all.

Unpinned-plan migration

requireModelValidationAuthority throws on plans with no pinned authority, so plans from finding-helpers, from the workflow, or persisted before this change would have hard-failed record_result and confirm.

Legacy unpinned plans are adopted as self with a recorded validationAuthorityMigration. That preserves exactly what those plans always were — model-asserted with no server facts — and never grants them a stronger claim. It does not bypass the per-payload resolver.

Tests

New tests/integration/validation-authority-execution.test.ts, 9 cases against a real database, real authorization ledger, and real DatabaseDurableDecisionRepository/authorizeAction — no injected approval literals:

  • auto with a genuinely minted-and-consumed approval, verified by reading it back
  • strict destructive admitted only by an exact operator approval; refused with none; refused with a mismatched approval
  • unauthorized target refused
  • the riskiest line, directly: destructive payload refused under both auto and self when the resolver denies, is absent, or throws
  • self model-decision threading
  • recovery (previously zero coverage): reload the persisted pin, refuse a corrected mode, resume under the pinned mode

The two tool tests that had been pinned to yolo — the one mode that bypasses the resolver entirely — now run under self.

Verification

Check Result
pnpm typecheck clean
pnpm test:unit 201 passed
pnpm test:integration 1176 passed, 7 skipped
pnpm test:evals 859 passed
pnpm test:packages 50 passed
pnpm build exit 0
biome on 11 changed files clean

No live or paid model calls.

Rebase note

The branch sat 107 commits behind. A commit-by-commit rebase conflicted repeatedly — main had reformatted the same files and renamed the eval row's authorityMode to requested/effective — so the cumulative diff was squash-applied onto origin/main as one commit. Two conflicts, both in the eval foundation, resolved by keeping main's per-quartet matched-row validation and v2 scorer/manifest versions while folding in the branch's requested/effective parity check. No redaction conflicts arose and nothing removed by #215 was reinstated.

Deliberately deferred

The eval manifest still declares executionAdmission: blocked by #100. Unblocking paid admission is a gated operations decision under the eval honesty rules, not a code change — flip it through the documented gate once this lands.

No UI control was added for the new project setting; it is settable through the project settings API and the setup path.

🤖 Generated with Claude Code

justsml and others added 2 commits September 13, 2026 17:55
Squash the PR #206 integration branch onto current origin/main (107
commits ahead) and reconcile the eval row schema: main's per-quartet
matched-row validation now reads `effectiveAuthorityMode`, and the
requested/effective parity check folds into the same pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The authority model shipped with no production resolver: the executor's
resolveExecutionAuthority was an optional dependency that only a test stub
ever supplied, so every non-yolo payload denied with
server-authority-not-resolved and no mode but yolo could execute.

Add src/server/validation-plans/authority-resolver.ts as the production
resolver. Per payload it resolves target authorization coverage from the
ledger, derives task scope from that authorization, resolves a containment
policy snapshot whose content-addressed id serves as both the policy
decision and the containment snapshot, creates the Tool Run, and settles
the durable approval: strict requires an exact operator approval covering
validationPayloadApprovalIntent(planId, payloadId), auto mints and consumes
its own exact one-time approval, self claims none. It never throws; every
missing fact becomes a denial record.

Wire it through both public seams. All four executing tool actions and the
validation plan workflow now construct the resolver, and both accept a
typed model decision (modelProceed / modelConfirm) so self is reachable
without widening model authority. executeMastraSecurityAction now stamps
the real selection into Tool Run metadata, and finding-helpers pins the
authority on the plans it creates.

Harden the executor so a throwing resolver denies the payload instead of
aborting the run - executionApprovalGranted returns true for auto and self
and relies entirely on this per-payload guard.

Operators select the mode as project.settings.validationAuthorityMode,
plumbed through the chat route into run metadata. Absence means strict; the
mode is never inferred from approvalMode.

Plans persisted before this contract carry no pin and would have hard-failed
record_result and confirm. They are adopted as self with a recorded
validationAuthorityMigration, preserving their original model-asserted
semantics without granting a stronger claim.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`pnpm check` now exits 0. Fixes organizeImports in trajectory-export,
authority-resolver and the two validation-authority test files, plus
formatting in security-actions/execution.ts.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@justsml
justsml merged commit 8078339 into main Sep 16, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make validation authority explicit and measure model guardrails

1 participant