Add transactional approval and durable execution recovery

Status: source/reviews and default-off sandbox deployment are complete; remaining live worker/recovery/queue/rollback evidence is consolidated into #128 after #166/#182 restore the canonical Card/Task baseline
Tags: enhancement, assistant, backend, infra, testing, data, human, P1
Depends on: #123 persistence/repository source prerequisite is satisfied; #124 resources are deployed default-off; live acceptance depends on the #128 HUMAN matrix after #166 A/B/C → #182 preflight → D → cleanup and one successful ordinary all-off deployment
Blocks: no source implementation — #126/#127 are shipped; #124 remains a live-acceptance dependency for closing #128/#122
Next owner: #166 orchestrator/credentialed operator for storage sequencing, then the authorized #128 HUMAN rollout owner with On-Call; PM closes #124 when its exact cross-posted criteria pass
Resume condition: #166/#182 finish the reviewed live sandbox sequence, canonical writers reopen on the final Tasks schema, temporary phase controls are removed, and normal all-off OIDC deployment succeeds. Then #128's text/todo, execution readiness/recovery/queue observation, alarm canary, and reverse rollback supply #124's live evidence; do not run a separate #124 provider/Telegram canary.
Architecture: _docs/CONVERSATIONAL_AGENT_PLUGIN_ARCHITECTURE.md at 4e5402c

Current lifecycle evidence

DataOps commit 840dd471a522b9275a62174185076eb6d1072080 is integrated and retains its Tester/PM-accepted transactional approval, lease, recovery, reconciliation, redaction, and infrastructure contracts. The least-privilege deploy-role repair that originally unblocked the retained failure queue was credentialedly applied and verified effective.

Normal push run 31545689094 at 5ac8b39c39e358cdec41b192bad984b8a46a2daa created the #124 execution worker function/role, Stream event-source mapping, retained encrypted failure queue, generated recovery and health-pulse EventBridge rules/permissions, and related runtime configuration; CloudFormation reached UPDATE_COMPLETE, runtime seeding completed, and deployed smoke passed without the former sqs:CreateQueue denial. All conversational controls were false, so the schedules and execution path correctly remained inactive. Run 31711997388 later reconfirmed the stable normal all-off OIDC deploy, seed, and smoke path.

Default-off deployment proves resource ownership and deploy permission, not live leasing/recovery/effect behavior. No HUMAN execution, queue/alarm canary, reconciliation, provider call, or switch rollback has run. #128 is the sole live-evidence owner because its accepted todo journey exercises this execution substrate end to end and its observation/rollback stages cover the shared recovery/queue/readiness contract.

#166 is the active sequencing boundary. The #128 todo canary would create a Task in the transitional table immediately before its approved deletion, so no #124 live acceptance starts until #166/#182 complete Phase D and cleanup.

Product decisions

Keep this slice channel- and domain-independent. It proves the safety boundary with deterministic fake renderers, authorization checks, executors, and reconcilers only. It does not call Telegram, z.ai, task mutation APIs, or Typefully.

Use the conversational state table and repositories introduced by #123. Add one DynamoDB Stream-triggered execution worker and one scheduled recovery invocation of that same worker. Do not add SQS or a general workflow engine in this slice.

Use one immutable proposal version, one or more channel presentations, and one deterministic execution-attempt ID per proposal version. Approval creates no new semantic content: the worker can execute only the stored ProposalSpec.

Use conservative crash handling. Once a worker durably records that provider dispatch is starting, a lost response is potentially an external success. Recovery may retry only with the same provider idempotency key, or may perform a read-only correlation lookup. Otherwise the attempt becomes outcome_unknown and requires an explicit admin resolution. It is better to surface uncertainty than create a duplicate effect.

Scope

Deterministic proposal and presentation service

Add a core service that:

validates a normalized ProposalSpec from Define conversational state, proposal records, retention, and recovery #123 and canonicalizes it with stable key ordering and explicit normalization rules;
stores a SHA-256 canonical payload hash;
renders a deterministic preview through an injected renderer and stores a SHA-256 rendered-view hash;
rejects nondeterministic output, omitted semantic fields, unknown permissions, unavailable executor builds, or hash mismatch;
creates a ProposalPresentation with a cryptographically random opaque token of at least 256 bits, stores only its SHA-256 hash, binds it to the exact actor identity, conversation, channel binding, proposal version, rendered view, and expiry, and defaults expiry to 30 minutes;
revokes older active presentations when a revision creates a new proposal version;
never accepts proposal content, target data, permissions, or provider parameters from callback/action payloads.

Test time, token generation, renderer, identity/authorization lookup, canonical-target revision lookup, executor registry, and reconciler are injected dependencies.

Atomic approval claim

Approval receives only the opaque presentation token plus trusted request provenance. It hashes the token, loads the referenced server-side records, and performs all read checks before attempting a transaction.

Immediately before the transaction, recheck:

the actor is authenticated and maps to the active identity bound to the presentation;
the actor still has the proposal's core permission;
the conversation and channel binding match the trusted request provenance;
the presentation is active, unexpired, and bound to the current rendered-view hash;
the proposal is presented, not superseded/canceled/expired/conflicted/claimed, and its plugin build, schema, policy, payload, and rendered-view digests match;
the capability-scoped executor build referenced by the proposal is still available;
the canonical target still has the proposal's stored base revision.

If the base revision changed, conditionally mark the unclaimed proposal conflicted, revoke its active presentations, create no execution attempt, and return a conflict that requires a new proposal. Authorization, actor, channel, expiry, hash, or build mismatch fails closed and creates no attempt.

One TransactWriteItems call must then:

condition-check the still-active identity/permission, conversation/channel binding, and canonical target revision records;
change exactly one matching presentation from active to consumed;
change exactly one matching proposal from presented to claimed;
insert one queued ExecutionAttempt if absent; and
append a minimal approval audit event if Define conversational state, proposal records, retention, and recovery #123 stores audit events in the same transactional table.

Every item uses condition expressions for the values checked above. The attempt ID and idempotency key are deterministic from the proposal-version identity and immutable payload hash, not from a callback delivery. A retried or simultaneous approval that loses the transaction reads and returns the already stored attempt/result when it belongs to the same proposal; it never queues a second attempt.

The approval path returns a normalized execution_pending interaction after a successful/already-successful claim. It never invokes an executor inline.

Capability-scoped execution and authorization recheck

Define a narrow executor contract keyed by capability and exact build digest. The worker supplies the stored proposal spec, attempt ID, stable idempotency key, and an abort/deadline signal. It does not supply arbitrary database/model/channel clients.

Before any effect, the worker must recheck active identity binding, current core permission, proposal/attempt relationship, exact executor build availability, immutable hashes, and the canonical target base revision. A revoked identity/permission, missing build, corrupt spec/hash, or changed target is a proven no-write failure: record failed_safe with a bounded redacted reason and do not call the executor. A target change after approval does not rewrite or silently refresh the proposal.

The fake executor records calls by idempotency key and can simulate:

success with a safe effect receipt;
a proved no-write error;
timeout/lost response before or after applying the fake effect;
provider-idempotent replay;
correlation lookup reporting applied, not_applied, or unknown;
operator-reconciliation-only behavior.

Result receipts contain only safe IDs, hashes, timestamps, and bounded redacted metadata. Secrets, action tokens, proposal bodies, and provider payloads are prohibited.

Leasing, dispatch boundary, and recovery

An ExecutionAttempt follows only these transitions:

queued -> executing -> succeeded
                    -> failed_safe
                    -> outcome_unknown
outcome_unknown -> succeeded          (authoritative read-only reconciliation)
                -> failed_safe        (authoritative read-only reconciliation)
                -> manually_resolved  (admin records applied/no-effect; no executor call)

succeeded, failed_safe, and manually_resolved are terminal. A terminal attempt is never executed again.

The worker:

claims an eligible queued attempt with one conditional update, recording a unique lease owner, lease expiry, incremented lease generation, bounded attempt history, and current executor build;
permits only the current unexpired lease generation to record dispatch or a result;
records a durable dispatch_started_at immediately before the external/fake executor call;
records success or safe failure with a conditional update and emits a normalized status/result event;
treats a stale worker's late result as non-authoritative and routes it through reconciliation rather than overwriting newer state.

The recovery invocation queries the #123 due-work index by nextActionAt; it must not scan the table. It handles:

queued attempts whose Stream delivery was missed by making them eligible for normal leasing;
expired executing leases with no dispatch marker by returning them to queued;
expired dispatched attempts in provider_idempotency mode by leasing and retrying with the same key;
expired dispatched attempts in correlation_lookup mode by running read-only reconciliation first, then recording succeeded, failed_safe, or outcome_unknown from the authoritative result;
expired dispatched attempts in operator_reconciliation_only mode by recording outcome_unknown without another executor call.

Infrastructure/transient failures before dispatch use bounded exponential backoff and a configured maximum pre-dispatch lease count. Exceeding it becomes failed_safe only when the system can prove no effect was attempted. outcome_unknown is never blindly requeued.

Status and reconciliation API

Add authenticated backend routes consistent with existing browser/portal authentication:

GET /api/conversational/execution-attempts/{id} returns a redacted status/result only to the owning authorized actor or an admin;
POST /api/conversational/execution-attempts/{id}/reconcile is admin-only, accepts no provider result from the caller, and invokes only the attempt's registered read-only reconciler;
POST /api/conversational/execution-attempts/{id}/resolve is admin-only and valid only for outcome_unknown; it records effect_applied or no_effect, a required bounded reason, actor, timestamp, and safe receipt, transitions to manually_resolved, and never invokes the executor.

All mutation routes require the current attempt revision and use conditional writes. Duplicate reconciliation/resolution requests return the stored state. Non-admin, wrong-owner, stale-revision, malformed, secret-bearing, or non-unknown resolution requests fail closed.

Atomic invariants

A proposal version has at most one execution attempt and one stable idempotency key.
A presentation can be consumed at most once; consuming one presentation globally claims the proposal, so every sibling presentation becomes unusable.
There is no reachable state where a presentation is consumed or a proposal is claimed without the corresponding queued attempt.
There is no reachable state where an attempt exists for an unclaimed proposal.
The executor receives exactly the stored spec whose payload and rendered view were approved.
Only one unexpired lease generation may dispatch or finalize an attempt.
No executor call occurs from the approval request.
No automatic executor call occurs after an operator-only uncertain outcome.
Authorization is checked both at approval and immediately before execution.
A duplicate callback, Stream record, scheduled recovery event, Lambda retry, or stale worker result cannot create a second effect or overwrite terminal truth.

Acceptance criteria

Deployment and live-closure criteria

The accepted Add transactional approval and durable execution recovery #124 source and least-privilege deploy-role repair are integrated/effective; normal CI creates the worker, Stream mapping, recovery/health schedules, retained encrypted failure queue, IAM/configuration, reaches terminal stack success, seeds, and passes deployed smoke with execution off.
Replace the transitional Tasks table with the final cardId schema #166 A/B/C, Gate canonical Card lifecycle on the empty production Cards table #182's no-replacement Cards preflight, D, and cleanup complete; canonical writers reopen, temporary phase controls are deleted, and a following normal all-off deployment succeeds before any live Add transactional approval and durable execution recovery #124/Roll out and cut over the Telegram conversational MVP #128 execution.
[HUMAN via Roll out and cut over the Telegram conversational MVP #128] The text + todo stage proves proposal approval returns pending, exactly one queued attempt is claimed, duplicate approval converges, the Stream worker produces exactly one actor-owned Task/result, and no inline approval executor or duplicate effect is observed.
[HUMAN via Roll out and cut over the Telegram conversational MVP #128] Bounded readiness/observation proves the execution worker, Stream mapping, recovery and health schedules, due-work query path, result delivery, runtime IAM/configuration, and redacted status/audit surfaces operate without scans, leaked proposal/token/provider content, or unauthorized access.
[HUMAN via Roll out and cut over the Telegram conversational MVP #128] The retained encrypted failure queue and its alarm path are verified with sanitized state/count evidence and no payload publication or purge. Any synthetic failure canary requires the explicit Roll out and cut over the Telegram conversational MVP #128 HUMAN gate and must prove no real provider/domain effect.
[HUMAN via Roll out and cut over the Telegram conversational MVP #128] Switch rollback disables new leasing/execution through normal OIDC while preserving queued/attempt/result truth, never retries an unknown effect, never overwrites terminal state, and requires no manual Lambda/IAM edit or destructive queue/data action.
Roll out and cut over the Telegram conversational MVP #128 cross-posts exact sanitized links for the criteria above. PM verifies no separate Add transactional approval and durable execution recovery #124 provider/Telegram canary, no fabricated reconciliation result, and no missing live criterion; Add transactional approval and durable execution recovery #124 may then close even if unrelated Roll out and cut over the Telegram conversational MVP #128 media/Typefully stages remain open.

Test scenarios

Exact preview binding

Given semantically identical objects with different key order, and objects differing in destination, content, permission, policy/schema/build digest, source revision, base revision, or expiry
When the service canonicalizes and renders them
Then key order produces the same hash, every semantic change produces a different hash, the preview is deterministic, and execution receives only the stored approved spec.

Concurrent approval

Given one active presentation and 25 simultaneous approval requests plus duplicate delivery retries
When all requests run against DynamoDB Local
Then one transaction consumes/claims/queues, one attempt ID exists, all successful/idempotent responses refer to it, and the fake executor is not called by the request path.

Wrong or stale action

Given wrong actor/channel, disabled identity, revoked permission, expired/revoked/consumed token, superseded proposal, changed policy/schema/build/hash, and changed base revision
When approval is attempted
Then no attempt is queued; target drift marks the proposal conflicted/revokes controls; every other case fails closed without exposing record existence.

Worker race and stale lease

Given duplicate Stream records, two workers, an expired lease, and a late result from the old worker
When they race to lease and finalize the same attempt
Then only one lease generation may dispatch/finalize, the late result cannot overwrite state, and the fake effect ledger contains at most one effect.

Crash boundaries

Given deterministic crashes (a) after approval transaction, (b) after lease before dispatch marker, (c) after dispatch marker before fake call, (d) after fake effect before response, and (e) after response before result write
When Stream retries and scheduled recovery run
Then (a) is eventually leased, (b) safely returns to queued, and (c-e) follow the declared provider-idempotency/correlation/operator-only policy without blind duplicate execution.

Delivery modes

Given identical crash cases for all three delivery modes
When recovery runs
Then provider-idempotent mode reuses one key, correlation mode performs a read-only lookup first, and operator-only mode becomes outcome_unknown with zero automatic retries.

Authorization change after approval

Given a claimed queued attempt whose actor is disabled, permission is revoked, executor build disappears, or base revision changes before dispatch
When the worker rechecks authorization
Then the fake executor call count remains zero and the attempt becomes a redacted failed_safe.

Reconciliation and manual resolution

Given outcome_unknown attempts and admin/operator/wrong-owner callers
When status, reconcile, and resolve routes are called concurrently or repeatedly
Then owner/admin status access is redacted, only admin may reconcile/resolve, the reconciler is read-only, one conditional resolution wins, repeat calls return stored truth, and no route calls the executor.

Infrastructure and failure delivery

Given the SAM template
When template assertions, sam validate, and sam build run
Then the correct table Stream invokes only the execution worker, scheduled recovery invokes the same worker in recovery mode, failed Stream batches reach the declared failure destination, worker/recovery IAM is limited to required table/index/stream/log resources, and the API function cannot assume worker-only permissions.

Regression and redaction

Given existing backend APIs and errors containing secret-like values
When the full backend suite runs
Then existing behavior passes unchanged and responses, logs, audit events, attempt receipts, and failure-destination payloads expose no secrets, opaque tokens, or proposal bodies.

Required verification

Software Engineer and Tester must run:

npm --prefix backend test
npm --prefix backend run typecheck
npm --prefix backend run build
make sam-validate
make sam-build

Also run focused DynamoDB Local tests for the transaction/race/lease/crash scenarios above, focused route authorization tests, and static SAM assertions for resources, event filters, failure handling, environment variables, and IAM. Use injected clocks and deterministic fake executors/reconcilers; automated tests make no external network calls or writes.

No portal UI changes are in scope, so Playwright and screenshots are not required unless implementation unexpectedly changes a web surface; Tester must state that rationale.

Infrastructure details

Reuse the Define conversational state, proposal records, retention, and recovery #123 conversational state table, keys, due-work index, encryption, PITR, retention policies, and export/restore boundary.
Enable DynamoDB Streams with the smallest view needed by the worker and filter the event source to execution-attempt records that become queued.
Add a separate execution-worker Lambda rather than granting external execution responsibilities to the public backend function.
Add bounded batch/retry settings, partial batch response, and an encrypted retained SQS failure queue for exhausted Stream delivery. This queue is diagnostic dead-letter storage only, not a work queue or orchestration layer.
Invoke the same worker on a short EventBridge recovery schedule with an explicit recovery event; do not add a second recovery implementation.
Grant the worker only required read/conditional-write/query/transaction permissions for conversational execution records and its due-work index. Grant Stream read permissions only through the event source mapping. Fake executors require no provider secrets or unrelated tables.
Grant the backend only the conversational transaction/status operations it needs. The public API must not receive worker-only provider permissions.
Configuration covers presentation TTL default, lease duration, pre-dispatch retry/backoff bounds, due-work bucket/index names, and fake executor enablement. Use safe defaults and validate bounds at startup.
Deployed resources are SAM/CloudFormation-owned; local table/event setup remains test/local-only.

Dependencies and closure ownership

#123's persistent records, conditional repositories, TTL/retention, token-hash lookup, unique attempt identity, and due-work index are satisfied source prerequisites. They are historical implementation context, not a current blocker. #121 and #125-#127 register/use the accepted channel, model, todo, and Typefully implementations; no source change is required in #124.

#128 owns all remaining live acceptance and public-safe evidence. #124 does not require every #128 capability to complete: its closure subset is the end-to-end transactional todo execution, worker/recovery/readiness/queue observation, and execution switch rollback listed above. Voice/photo and Typefully-only outcomes remain their own child/rollout gates. If a real outcome_unknown occurs, only the owning domain rollout may use its accepted reconciliation process; #124 closure does not authorize fabricating provider truth or invoking admin resolution merely to exercise a route.

#166/#182 are sequencing dependencies only because the shared todo canary writes the Tasks table. They do not change #124's conversational-state schema or justify migration/compatibility machinery here.

Out of scope

Telegram callbacks/keyboards, voice notes, images, channel rendering, or group behavior.
z.ai, Groq, Typefully, task, GitHub, or any other real provider/domain mutation.
Todo or social-post schemas and plugin registration.
Model calls, skill_load, skill_invoke, plugin selection, or general agent behavior.
Web proposal UI, cross-channel approval, notification delivery, or cancellation of an executing external action.
Multiple effects, batches, partial success, compensating transactions, arbitrary workflow graphs, SQS orchestration, Step Functions, or a general flow DSL.
Retrofitting/replacing legacy AssistantJob approval routes.
Live provider reconciliation runbooks or real-provider smoke tests; those belong to the domain plugin and rollout issues.
Modifying ../dtc-operations, ../datatasks, or ../podcast-assistant.

GROOMED

This issue is ready for implementation after #123 closes.

Key product decisions:

Keep M4 channel/domain independent and prove it with fake renderers, executors, authorization, and reconcilers only.
Reuse the Define conversational state, proposal records, retention, and recovery #123 conversational state table; add one Stream-triggered worker and invoke that same worker on a recovery schedule. No general workflow engine or SQS work queue.
Approval is one DynamoDB transaction with authorization/base-revision condition checks plus presentation consume, proposal claim, deterministic attempt creation, and transactional audit when supported by Define conversational state, proposal records, retention, and recovery #123.
Recheck authorization and target revision immediately before dispatch.
Mark dispatch durably before the call. After that boundary, retry only with the same provider idempotency key or perform read-only correlation lookup; otherwise surface outcome_unknown.
Keep manual resolution admin-only, revision-checked, audited, and incapable of invoking the executor.
Require deterministic DynamoDB Local concurrency/crash tests and SAM/IAM/event-source assertions in addition to the full backend verification.

No real Telegram, z.ai, task, Typefully, or other external mutation is in this slice.

Software Engineer handoff

Implemented the groomed #124 transactional approval and durable execution-recovery slice with injected fake executors only.

Delivered

Deterministic strict-JSON ProposalSpec canonicalization and preview rendering with SHA-256 payload/view binding.
256-bit opaque presentation tokens with only their SHA-256 hashes stored, configurable 30-minute action expiry, exact actor/channel/conversation/view binding, and superseded-control revocation.
Approval by opaque token plus trusted provenance only.
One production TransactWriteItems approval claim with current user, active identity, permission, conversation, channel, optional target revision, presentation, proposal, deterministic attempt, audit, and cleanup-link conditions/writes.
Deterministic attempt IDs/idempotency keys and idempotent 25-way concurrent approval; approval never calls an executor.
Capability/build-scoped fake executors and a separate worker with conditional lease generations, durable dispatch markers, pre-effect user/identity/permission/hash/build/target rechecks, deadlines, bounded safe receipts, and stale-result rejection.
Indexed scheduled recovery with bounded exponential pre-dispatch backoff and all three delivery modes:
- provider-idempotent replay uses the same key;
- correlation mode performs read-only lookup;
- operator-only uncertainty becomes outcome_unknown.
Authenticated owner/admin redacted status plus admin-only, revision-checked read-only reconciliation and manual resolution. Caller-supplied provider results and secret-bearing reasons are rejected; manual resolution never invokes the executor.
Separate SAM worker with filtered DynamoDB Stream input, partial-batch response, bounded retries, the same worker on a recovery schedule, encrypted retained SQS failure destination, bounded environment configuration, and table/index-scoped IAM.
Existing assistant-job approval remains unchanged. No Telegram, z.ai, todo mutation, Typefully, model, or real provider behavior was added.

Changed files

backend/src/conversation/{execution,executionDefaults,executionRepository,executionWorker,types}.ts
backend/src/execution-worker-handler.ts
backend/src/routes/conversationalExecution.ts
backend/src/router.ts
backend/tests/conversational-execution.test.ts
infra/template.full.yaml
tests/infra/test_template.py
Makefile
backend/README.md

Verification

Dedicated Add transactional approval and durable execution recovery #124 DynamoDB Local safety suite: 10/10 passed
- includes 25 simultaneous approval requests;
- 25 duplicate worker deliveries;
- exact production approval transaction-shape assertion;
- wrong/stale provenance and target conflict;
- disabled user/permission pre-dispatch denial;
- lease race, pre-dispatch crash/backoff, and stale result rejection;
- provider-idempotent, correlation, and operator-only post-dispatch recovery;
- owner/admin status, reconciliation, manual resolution, revision, redaction, and no-executor assertions.
Broader focused conversational/runtime/export/archive suites: 46/46 passed.
Full backend regression on the final tree: 1,287/1,287 passed, 179/179 suites.
Backend typecheck: passed.
Backend build: passed.
Infrastructure assertions: 25/25 passed.
Planning-doc assertions: 7/7 passed.
sam validate: passed.
Final sam build: passed for both ConversationalExecutionWorkerFunction and BackendFunction.
git diff --check: passed.

No external network calls, production writes, commit, push, or issue close were performed. No portal UI changed, so Playwright/screenshots are not applicable per the groomed issue.

READY FOR TEST

CHANGES REQUESTED

The broad regression workflow passes, but independent safety probes found multiple blockers in the exact approval/execution contract.

Blocking findings

Rendered-view binding omits semantic proposal fields (AC 1).

The fake renderer/view hash includes effect/content/target/destination/permission/base revision, but not policy digest, schema digest, plugin build digest, source references, or expiry. An independent probe changed each of those fields: the canonical payload hash changed every time, while the rendered-view hash stayed identical every time. The groomed exact-preview scenario requires every listed semantic edit to invalidate the approved view. Bind the view to the complete canonical proposal (for example by including the canonical payload hash in the deterministic rendered envelope), and assert both hashes for every semantic field.
The worker does not recheck the exact approved identity/channel binding (AC 7).

ExecutionAttempt stores no identityBindingId or channel-conversation binding. preDispatchCheck accepts any active identity belonging to the actor and never reloads the approved channel binding. In an isolated no-write probe I approved through identity-exact, revoked that identity, left a second identity for the same user active, and ran the worker. The fake executor was called once and the attempt became succeeded. The exact presentation-bound identity/channel must be stored on the attempt and rechecked immediately before dispatch.
Unsafe executor receipts are persisted and exposed without validation/redaction (AC 7 and 12).

finalizeAttempt writes the executor receipt directly without applying validateConversationalRecord/safe-receipt validation, and publicAttempt returns the complete receipt. An injected fake executor returned a bearer-shaped receipt ID and metadata.apiKey = SYNTHETIC_SECRET_MARKER; both values were persisted, and an independent publicAttempt probe reported leaksBearer:true and leaksSecret:true. Validate a narrowly allowlisted receipt before any write, reject/redact secret-shaped keys and values, and expose only explicitly safe fields.
An expired lease generation can still finalize terminal truth (AC 8).

markDispatchStarted checks leaseExpiresAt > now, but finalizeAttempt checks only status/owner/generation. An independent local probe leased an attempt through 12:01, then finalized it at 12:02 without any renewal; the update succeeded as succeeded. This violates the invariant that only the current unexpired lease may record a result. Add lease-expiry/current-revision conditions to finalization and cover the expired-owner-before-reclaim race.
The generated worker policy source lacks required Users-table read permission (AC 13).

preDispatchCheck calls getUser, which reads DataOpsUsersTable, but ConversationalExecutionWorkerFunction IAM lists only the conversational table and its two indexes. In deployment this reaches GetItem without authorization before every fake dispatch. Grant only dynamodb:GetItem on the exact Users-table ARN and add a static/generated-template assertion that proves the dependency while keeping all other tables unavailable.
Approval controls and transaction conditions are not fully closed over the approved state (AC 2–4).
- Approving one of two presentations left the sibling presentation stored as active in an independent probe. It is indirectly unusable because the proposal is claimed, but the required global control revocation/state is not recorded.
- Supersession inserts the new proposal/presentation and then revokes old controls one by one; a crash can leave an old proposal/control active.
- The production transaction conditions payload/view hashes but not the stored nested policy/schema/build/permission/expiry fields that were prechecked. A mutation between read and transaction can therefore queue an attempt and defer corruption detection to the worker instead of failing approval closed.
Make supersession/control invalidation durable and ensure the approval transaction conditions cover the complete immutable execution envelope required by the issue.
The required transactional/crash-boundary test evidence is incomplete.

The 25-way approval test runs with NODE_ENV=test, and atomicApproval then uses its sequential Dynalite compatibility branch rather than TransactWriteItems. A separate fake-client assertion inspects the production command shape, but no 25-way test executes the real transaction path against DynamoDB Local as groomed. The suite also does not deterministically exercise every named crash boundary, especially after dispatch marker/before call and after response/before result write. Add a real transaction-capable local test path and explicit crash hooks/assertions.

Passing evidence

NODE_ENV=test SKIP_AUTH=true node --import tsx --test backend/tests/conversational-execution.test.ts — exit 0, 10/10, 1 suite.
npm --prefix backend test — exit 0, 1,287/1,287, 179 suites.
npm --prefix backend run typecheck — exit 0.
npm --prefix backend run build — exit 0.
uv run --with pytest python -m pytest tests/infra -q — exit 0, 25/25.
uv run --with pytest python -m pytest tests/planning_docs -q — exit 0, 7/7.
make sam-validate — exit 0; template valid.
git diff --check — exit 0.
make sam-build was invoked and remained active in the custom CopySource/build step when this blocking verdict was posted; the source-template IAM defect above is independently conclusive and must be fixed before retest.

Acceptance criteria, in order

FAIL — canonical payload binding is deterministic, but rendered-view binding omits listed semantic fields.
FAIL — token size/hash/expiry and supersession happy path work; sibling/global revocation and crash-atomic supersession do not.
FAIL — read checks exist, but exact transaction closure over the immutable spec is incomplete.
FAIL — one production transaction is constructed, but required conditions/real concurrent transaction verification are incomplete.
PASS with required retest — deterministic ID and current 25-way compatibility path yield one attempt.
PASS — approval returns execution_pending and does not invoke an executor.
FAIL — exact identity/channel recheck and safe-receipt enforcement are missing.
FAIL — 25 workers produce one fake effect and stale generations are rejected after reclaim, but an expired current generation can finalize.
PASS — scheduled recovery uses GSI2 queries, not scans, and does not requeue outcome_unknown.
PASS — all three delivery modes follow their declared retry/reconciliation behavior in the focused suite.
PASS — owner/admin route isolation, revision checks, caller-result rejection, and no-executor manual resolution pass.
FAIL — unsafe receipt content is persisted/exposed; audit transitions are otherwise bounded in the tested happy paths.
FAIL — Stream filter, shared scheduled worker, retained SSE queue, and scoped conversational resources are present, but required Users-table GetItem IAM is absent.
PASS — full backend regressions pass and legacy assistant-job approval is unchanged.

Playwright and screenshots are not applicable because no portal page, browser route, or operator UI changed. All adversarial probes used local Dynalite and injected fake executors only. No live provider call, production write, deploy, source-repository edit, commit, or push was performed.

Software Engineer re-handoff — tester findings resolved

Resolved all seven blocking findings from the prior Tester review.

Fixes

Complete rendered-view binding
- The deterministic preview envelope now binds the complete canonical proposal hash and exact renderer build digest.
- Tests independently mutate destination, content, permission, policy, schema, build, sources, base revision, and expiry and require both canonical and rendered-view hashes to change.
Exact approval provenance at dispatch
- Attempts persist the approved identity binding ID/channel/user and channel binding ID/conversation key.
- Approval transaction conditions the exact identity and channel binding IDs.
- The worker reloads and verifies those exact records immediately before dispatch; another active identity for the same actor cannot substitute.
Safe receipts only
- Receipt IDs, hashes, timestamps, metadata keys, scalar values, sizes, and secret/bearer shapes are validated before persistence or reconciliation.
- Unsafe executor/reconciler receipts become bounded outcome_unknown results without storing the receipt.
- Public status serializes only a validated, explicitly safe receipt projection and omits unsafe legacy/corrupt receipts.
Unexpired current lease required
- Finalization now conditions exact status, revision, owner, generation, and leaseExpiresAt > now.
- A dedicated test proves an otherwise-current owner cannot finalize after lease expiry and before reclaim.
Worker Users-table permission
- The worker receives DATAOPS_USERS_TABLE and exact-ARN dynamodb:GetItem permission only.
- Source-template and generated-build assertions cover the dependency.
Atomic control closure
- New-version proposal/presentation/link creation and stale proposal/presentation invalidation are one production transaction.
- Approval consumes the selected presentation and revokes every active sibling presentation in the same transaction.
- Proposal claim conditions include payload/view hashes plus nested build, schema, policy, permission, and spec-expiry values.
Production transaction and crash evidence
- Added a transaction-capable 25-way production-path harness: 25 TransactWriteCommand attempts, exactly one winner.
- Added explicit deterministic coverage for all five boundaries:
  1. after approval transaction;
  2. after lease, before dispatch marker;
  3. after dispatch marker, before executor call;
  4. after fake effect, before response;
  5. after executor response, before durable result write.

Final verification

Dedicated Add transactional approval and durable execution recovery #124 suite: 17/17 passed, 1 suite.
Full backend regression: 1,294/1,294 passed, 179 suites.
Backend typecheck: passed.
Backend build: passed.
Infrastructure assertions: 25/25 passed.
Planning-doc assertions: 7/7 passed.
sam validate: passed.
sam build: passed for both ConversationalExecutionWorkerFunction and BackendFunction.
Generated SAM template contains the worker Users-table environment binding and scoped GetItem policy.
git diff --check: passed.

No prescribed final check failed. No external provider call, production write, deploy, source-repository edit, commit, push, or issue close was performed. No portal UI changed, so Playwright/screenshots remain not applicable.

READY FOR TEST

CHANGES REQUESTED

The consolidated fixes close every previously demonstrated runtime, redaction, lease, control, and IAM defect. One required verification gap remains: the 25-way “production transaction” test does not execute a DynamoDB transaction or its condition expressions against transaction-capable storage.

Sole blocker: production transaction concurrency is still simulated

admits one winner through the production transaction path under 25 simultaneous approvals passes a fake DynamoDBDocumentClient to atomicApproval. That fake:

records each TransactWriteCommand;
uses one process-local boolean (transactionClaimed) to fulfill the first call;
synthetically throws TransactionCanceledException for the other 24 calls;
does not evaluate any ConditionCheck, Update, or Put;
does not atomically mutate presentation, proposal, attempt, audit, relationship, identity, permission, channel, or target state.

The separate 25-way approvePresentation test does mutate Dynalite, but NODE_ENV=test selects the sequential compatibility branch instead of TransactWriteItems. Therefore the groomed requirement to run the 25 simultaneous approval race against DynamoDB Local and prove one complete transaction/attempt with no partial state remains unverified. The new test proves command construction plus its own boolean, not DynamoDB transaction behavior.

Smallest acceptable correction: run the 25 concurrent full approvePresentation calls against a transaction-capable DynamoDB Local instance with the production TransactWriteItems branch enabled, then independently read storage and assert:

all idempotent responses resolve to the same deterministic attempt;
selected presentation is consumed and every active sibling is revoked;
proposal is claimed;
exactly one queued attempt, audit event, and relationship link exist;
all actor/identity/permission/conversation/channel/target/spec conditions were actually evaluated;
no consumed/claimed state exists without the matching attempt, and no attempt exists for an unclaimed proposal.

A fake client with a manually selected winner is not sufficient for this safety boundary.

Original counterexamples — now passing

Semantic preview binding: independent mutations of policy digest, schema digest, source references, and expiry changed both canonical payload and rendered-view hashes.
Exact identity: after revoking the approved identity while leaving another actor identity active, execution became failed_safe; executor calls: 0.
Exact channel: replacing the stored approved channel-binding ID caused failed_safe; executor calls: 0.
Unsafe receipt: bearer-shaped ID plus secret-key/value metadata became receipt-free outcome_unknown; nothing unsafe was persisted or returned by publicAttempt.
Sibling approval controls: the sibling presentation became revoked.
Supersession: production command inspection shows new proposal/presentation/link plus old proposal/presentation invalidation in one TransactWriteCommand.
Expired lease: an otherwise-current expired generation could not finalize.
Crash boundaries: all five explicit boundaries passed with one stable idempotency key and no duplicate fake effect.
IAM/generated SAM: worker has DATAOPS_USERS_TABLE, exact Users-table dynamodb:GetItem, conversational table/index permissions, filtered Stream event, shared recovery schedule, and retained SSE failure queue.

Commands and evidence

PASS — NODE_ENV=test SKIP_AUTH=true node --import tsx --test backend/tests/conversational-execution.test.ts: 17/17, 1 suite.
PASS — focused original-counterexample probe: exact identity failed_safe/0 calls; channel mismatch failed_safe/0 calls; unsafe receipt outcome_unknown/no stored receipt/no API leak; sibling revoked.
PASS — independent semantic-hash probe: payload and view changed for policy/schema/source/expiry.
PASS — npm --prefix backend test: 1,294/1,294, 179 suites.
PASS — npm --prefix backend run typecheck.
PASS — npm --prefix backend run build.
PASS — uv run --with pytest python -m pytest tests/infra -q: 25/25.
PASS — uv run --with pytest python -m pytest tests/planning_docs -q: 7/7.
PASS — make sam-validate.
PASS — make sam-build; both worker and backend artifacts built, generated template inspected.
PASS — git diff --check.
FAIL — required transaction-capable 25-way DynamoDB Local execution is absent; current “production” harness simulates the winner itself.

Acceptance criteria

1–3: PASS.
4–5: NOT VERIFIED / FAIL GATE — production transaction shape is correct by inspection, but atomic conditions, partial-state impossibility, and concurrent idempotency are not executed against transaction-capable storage as required.
6–14: PASS based on focused probes, full regression, route isolation, crash/recovery tests, redaction checks, and generated SAM inspection.

Playwright and screenshots remain inapplicable because no portal page, browser route, or operator UI changed. No live provider call, production write, deploy, source-repository edit, commit, or push was performed.

Software Engineer re-handoff — real transaction concurrency proof added

Resolved the sole remaining Tester blocker without changing production behavior.

Genuine DynamoDB transaction test

Added:

npm --prefix backend run test:execution-transaction
backend/scripts/test-conversational-execution-transaction.sh
the transaction-capable case in backend/tests/conversational-execution.test.ts

The command starts a fresh amazon/dynamodb-local:latest in-memory container on
an ephemeral localhost port, waits for readiness, runs the test through
DYNAMODB_ENDPOINT, and removes the container on exit. It uses the repository's
supported DynamoDB Local image and adds no production dependency.

The test runs 25 concurrent full approvePresentation calls while selecting
the production TransactWriteItems branch. There is no fake client, manually
chosen winner, or sequential Dynalite compatibility path.

Independent strongly consistent reads prove:

all 25 calls resolve successfully to the same deterministic attempt and
idempotency reference;
the proposal is claimed;
the selected presentation is consumed;
both independently stored active sibling presentations are revoked;
exactly one queued execution attempt exists;
exactly one approval_claimed audit event exists;
exactly one proposal-attempt conversation relationship link exists and points
to that attempt;
claimed/consumed state and the complete attempt/audit/link records exist
together, with no partial transaction state.

Because the successful production transaction executes against DynamoDB Local,
its user, exact identity, permission, conversation, exact channel, target,
presentation, full immutable proposal-envelope, and absent-attempt/link/audit
conditions are evaluated by DynamoDB rather than simulated by test code.

The prior process-local boolean harness was removed.

Verification

Real transaction container command: 1/1 passed, 25 concurrent full
approvals, 0 skipped.
Ordinary focused suite: 16 passed, 0 failed, 1 intentionally skipped
container-only case.
Full backend regression: 1,293 passed, 0 failed, 1 intentionally skipped
container-only case; 1,294 total tests, 179 suites.
Backend typecheck: passed.
git diff --check: passed.
Container cleanup verified: no transaction-test container remained.

No production runtime behavior, external provider, production data, deployment,
commit, push, or issue state was changed.

READY FOR TEST

TEST PASS — AWS OIDC deploy-role fix

Reviewed isolated aws-infra commit 691666a9b31ae58eba57b009c4bbd0b422d32231 on branch issue-124-conversational-execution-deploy-role against DataOps commit 840dd47, the failed deployment run 30505675113, the #124 SAM resource graph, and the AWS service-authorization boundaries. No edits, commit, push, stack apply, or AWS mutation were performed.

Coverage verified:

SQS: the exact denied queue ARN dataops-v1-ConversationalExecutionFailureQueue-A0vcdnnKoH7b matches the new stack/logical-ID-scoped queue ARN pattern. Create, delete, attribute read/update, URL read, and tag list/add/remove actions are all present. The SAM queue uses SQS-managed SSE, so no KMS permission is needed.
Lambda worker: both unqualified and qualified generated worker-function ARN patterns were added to the existing narrowly scoped lifecycle/read/invoke/tag statement. This retains all existing function patterns and supports generated Lambda permission management for the recovery schedule.
Worker IAM role: the exact generated SAM role-name pattern was added to the existing create/delete, inline/attached-policy, assume-role, read/tag, and iam:PassRole lifecycle statement. Existing application roles remain unchanged.
Logs: both generated worker log-group ARN forms were added to the existing create/delete/describe/retention/tag statement. Existing log groups remain covered.
DynamoDB event-source mapping: CreateEventSourceMapping correctly uses Resource * because the AWS Lambda authorization reference exposes no resource type for create, and it is constrained with the supported lambda:FunctionArn condition to only this worker. Get/update/delete are scoped to eu-west-1/account mapping ARNs and carry the supported worker FunctionArn condition. ListTags/tag/untag are scoped to event-source-mapping ARNs in only this region/account; AWS does not expose lambda:FunctionArn for those tag actions, so this is the narrow effective resource boundary CloudFormation can use for generated UUIDs.
Recovery schedule: the existing stack-scoped EventBridge rule statement and the newly covered worker Lambda AddPermission path already cover the generated schedule; no broad EventBridge permission was added.
Existing resources: the patch is additive only, 45 insertions and 0 deletions in one template. Existing Lambda, IAM-role, log-group, EventBridge, DynamoDB, S3, and secret patterns/actions are preserved.

Commands and results:

PASS — sam validate --lint --template-file sandbox/dataops/template.github-actions.yaml: valid SAM/CloudFormation template.
PASS — uv run --with pytest python -m pytest sandbox/dataops/tests -q: 7/7.
PASS — git diff --check 804b745..691666a.
PASS — git diff --name-only 804b745..691666a: only sandbox/dataops/template.github-actions.yaml.
PASS — git diff --stat 804b745..691666a: 45 insertions, 0 deletions.
PASS — git status --porcelain=v1: clean.

The policy change is least-privilege within CloudFormation generated-name/UUID constraints and covers the complete #124 resource lifecycle identified by On-Call. It is ready for the credentialed operator integration/apply gate. A live policy evaluation and stack deployment remain intentionally outside this review and must be verified after the shared infrastructure stack is applied.

PM ACCEPTED — AWS deployment-role hotfix

Commit 691666a9b31ae58eba57b009c4bbd0b422d32231 is accepted for integration into DataTalksClub/aws-infra and credentialed application of the dataops-github-actions stack.

Product/operator decision

This is the smallest complete fix for the deployment failure reported by On-Call:

It changes only sandbox/dataops/template.github-actions.yaml and is additive: 45 insertions, no deletions.
The exact denied SQS queue name is covered by a stack/logical-ID-scoped ARN pattern, with only the queue lifecycle, attributes, and tag actions CloudFormation needs.
Worker Lambda, execution role/iam:PassRole, and log-group access extend the existing lifecycle statements using only the generated worker resource-name patterns.
Event-source-mapping create uses the AWS-required Resource: "*", but is constrained by the supported lambda:FunctionArn condition to this worker. Read/update/delete remain region/account/UUID scoped and worker-conditioned. Mapping tag actions use the narrow region/account resource boundary available for generated mapping UUIDs.
The existing stack-scoped EventBridge policy already covers the recovery schedule, so no redundant or broader EventBridge grant was added.
The queue uses SQS-managed encryption, so the patch correctly adds no KMS permission.
Existing application-resource permissions are preserved and no unrelated service, repository, or deployment behavior changes.

This balances simplicity with least privilege: one deploy-role policy update covers the complete #124 CloudFormation resource graph instead of iterating through predictable one-permission deployment failures.

Evidence

On-Call isolated the rollback to missing sqs:CreateQueue and identified the remaining worker resource lifecycle gaps.
Tester independently verified the exact denied ARN, Lambda/role/log patterns, event-source-mapping authorization boundaries, retained EventBridge scope, and absence of unnecessary KMS access.
PM reran uv run --with pytest python -m pytest sandbox/dataops/tests -q: 7/7 passed.
PM reran sam validate --lint --template-file sandbox/dataops/template.github-actions.yaml: valid.
git diff --check 804b745..691666a: passed; isolated worktree remains clean.

Required operator sequence

Integrate and push commit 691666a in DataTalksClub/aws-infra, preserving unrelated changes in the existing dirty checkout.
A credentialed operator applies the dataops-github-actions CloudFormation stack with CAPABILITY_NAMED_IAM and confirms UPDATE_COMPLETE.
Rerun the failed DataOps deployment for 840dd471a522b9275a62174185076eb6d1072080.
Relaunch On-Call to verify the deployed worker, Stream mapping, recovery schedule, encrypted retained failure queue, effective runtime IAM/configuration, and redacted logs.

The policy source is accepted; #124 deployment acceptance remains incomplete until the credentialed apply, DataOps redeploy, and On-Call production verification succeed.

AWS deploy-role source fix merged: https://github.com/DataTalksClub/aws-infra/pull/18. Automated review, 7/7 infra tests, SAM lint, and PM acceptance passed. Remaining gate: a credentialed AWS operator must apply the dataops-github-actions stack with CAPABILITY_NAMED_IAM; then rerun the failed DataOps deployment and On-Call verification.

On-Call follow-up from the #121 push:

Planning/infra validation passed.
Deployment failed again while creating ConversationalExecutionFailureQueue.
Exact failure remains AccessDenied for sqs:CreateQueue by dataops-github-actions-deploy.
CloudFormation reached terminal UPDATE_ROLLBACK_COMPLETE; seed and smoke were skipped.

aws-infra PR #18 is merged, but this repeated denial confirms the credentialed dataops-github-actions stack update is still not applied in AWS. Consequently the #124 worker, Stream mapping, recovery schedule, encrypted failure queue/redrive, worker IAM/environment, and logs remain unavailable for production acceptance.

No edits, manual deploy, or AWS mutation were performed by On-Call. Required next step remains: credentialed operator applies PR #18's infrastructure stack, rerun the DataOps deployment, then relaunch On-Call verification.

On-Call update — main `3365c2a`

The latest DataOps deploy workflow failed in the backend test gate before OIDC authentication or any AWS operation. It therefore provides no new evidence about the live role permission state and does not supersede the accepted #124 infrastructure source change or pending operator apply.

Canonical current-run diagnosis: #115 (comment)

On-Call update — main `98fdc1f`

The fresh deployment reached AWS after every repository-owned gate passed, then reproduced the known live #124 permission boundary:

workflow run 30526965133;
deploy job 90821197586;
OIDC configuration and deploy-side SAM build passed;
CloudFormation was denied sqs:CreateQueue for ConversationalExecutionFailureQueue;
the stack reached terminal UPDATE_ROLLBACK_COMPLETE;
seed and deployed-origin smoke were skipped.

This is not an application/test failure: the check job passed backend tests with 0 failures and 0 cancellations, typecheck, build, 346 Playwright tests, SAM validation, and SAM build. The denial confirms that accepted #124 policy source is still not effective on the live deploy role. The current combined operator source is DataTalksClub/aws-infra origin/main 2a1b95f (#124 + #129 + #130).

Canonical report: #115 (comment)

Required next step remains the credentialed named-IAM stack apply, effective-policy verification, then a fresh current-main deployment and On-Call production verification. No manual deploy or AWS mutation was performed.

On-Call update — main `4f10a5b`

Fresh current-main evidence again confirms the accepted #124 policy source is not effective on the live deploy role:

workflow run 30530804398;
deploy job 90833807176;
every repository-owned gate passed: backend 1,368 passed + 1 expected skip, typecheck/build, Playwright 352/352, SAM validate/build;
OIDC credentials and deploy-side SAM build passed;
CloudFormation was denied sqs:CreateQueue for ConversationalExecutionFailureQueue and rolled back to UPDATE_ROLLBACK_COMPLETE;
seed and deployed-origin smoke were skipped.

Canonical On-Call report: #106 (comment)

This is the known credentialed named-IAM apply boundary, not a code regression. The accepted combined aws-infra source remains origin/main 2a1b95f; a credentialed operator must apply and verify it before fresh normal CI/CD. No manual deploy or AWS mutation was performed.

On-Call update — main `c47c525`

Workflow run 30532699861 passed backend tests/typecheck/build, Playwright, SAM validation, and two SAM builds, then reproduced the live #124 boundary: sqs:CreateQueue was denied for ConversationalExecutionFailureQueue under the GitHub Actions deploy role.

CloudFormation reached terminal UPDATE_ROLLBACK_COMPLETE; no execution worker deployment or runtime smoke test completed. This is direct evidence that accepted combined deploy-role source at DataTalksClub/aws-infra 2a1b95f is still unapplied to the live role. A credentialed operator must apply the named-IAM stack; do not manually deploy the DataOps app.

Canonical #126 On-Call report: #126 (comment)

PM STATUS RECONCILIATION — code complete; production acceptance still blocked

The stale pending / Depends on: #123 header no longer described the shipped
state.

DataOps commit 840dd471a522b9275a62174185076eb6d1072080 is on main. Its implementation passed the final Tester review and PM acceptance. The implementation acceptance checkboxes are therefore complete.
The original rollout failure was the deploy role's denied sqs:CreateQueue. The least-privilege source fix passed Tester and PM, was later integrated into the combined deploy-role updates, and the combined named-IAM policy was applied and verified effective.
Later deployment evidence advanced past that original SQS denial and reached the Sponsor table ownership guard. The latest bounded run authenticated with OIDC, passed the preceding identity/mapping checks, and stopped before build/deploy on the missing canonical stack-ID tag.
That current application-deployment block is owned by Migrate Sponsor CRM GSIs in a protected stage-only workflow #136, which depends on Reconcile missing CloudFormation ownership provenance for Sponsor CRM table #140, which depends on the HUMAN CloudTrail evidence session in Close out the retained CloudTrail audit session record #141.

PM decision: keep #124 open and add human. Code and repository acceptance are
complete, but production deployment/runtime On-Call acceptance is not. After
#136 and its inherited #140/#141 provenance chain are resolved, a fresh normal
CI/CD deployment must verify the #124 execution worker, Stream mapping, recovery
schedule, retained encrypted failure queue/redrive behavior, effective runtime
IAM/configuration, status/reconciliation routes, and redacted logs. Only then
may #124 close.

No code, AWS, IAM, workflow, or environment change was made during this
reconciliation.

PM tracker reconciliation — runtime deployment handoff clarified

The accepted code on main is unchanged. Replaced the stale #141 dependency with #140 lifecycle acceptance plus #143 ownership remediation, added the exact On-Call owner and terminal-success deployed-smoke resume signal, and removed the premature human label while production verification remains. No implementation or criterion changed.

PM lifecycle reconciliation — default-off deployed; live closure delegated to #128

#124 remains open, but the old #136/#143 deployment block is complete.

Completed evidence

Accepted source commit 840dd471a522b9275a62174185076eb6d1072080 remains integrated with its full transactional approval, leasing, recovery, uncertainty, authorization, and redaction contract.
The least-privilege deploy-role repair is effective.
Normal run 31545689094 created the execution worker/role, Stream mapping, retained encrypted failure queue, recovery and health-pulse schedules/permissions, reached UPDATE_COMPLETE, seeded, and passed deployed smoke without the former SQS denial. All conversational controls were off, so execution/schedules correctly remained inactive.
Run 31711997388 reconfirmed the normal all-off OIDC deploy, seed, and smoke path.

This proves default-off resource deployment, not live leasing/recovery/effect behavior. No execution, queue/alarm canary, reconciliation, provider effect, or rollback has run.

Consolidated live owner and #166 sequence

#128 is the sole remaining live-evidence owner; #124 must not create a second Telegram/provider canary. Its text + todo stage exercises transactional approval, one queued attempt, Stream-worker dispatch, one Task/result, duplicate convergence, and result delivery. Its readiness/queue/alarm observation and switch rollback cover the shared recovery substrate.

That evidence waits for #166 A/B/C, #182's no-replacement Cards preflight, Phase D, and cleanup. Starting now would create the todo canary Task in the transitional table immediately before approved deletion. Resume after canonical writers reopen, temporary phase controls are gone, ordinary push deployment is restored, and one all-off run succeeds.

Exact #124 closure subset

PM may close #124 after #128 cross-posts sanitized PASS evidence for:

one end-to-end transactional todo approval/execution with duplicate convergence and exactly one effect/result;
worker, Stream mapping, indexed recovery/health schedules, IAM/configuration, status/audit redaction, and unauthorized-access boundaries;
retained encrypted failure-queue/alarm wiring with counts/state only, no payload publication or purge;
normal-OIDC execution switch rollback preserving queued/attempt/result truth and never retrying unknown effects.

A fabricated outcome_unknown, provider truth, or admin resolution is not required and is not authorized merely for closure. #124 can close when this subset passes even if unrelated #128 voice/photo/Typefully stages are still open. The human label now truthfully reflects the lifecycle.

No repository edit, test, AWS/IAM call, workflow dispatch, provider/Telegram action, credential use, flag change, queue action, data operation, import/export, or restore was performed.

Add transactional approval and durable execution recovery #124

Description

Add transactional approval and durable execution recovery

Current lifecycle evidence

Product decisions

Scope

Deterministic proposal and presentation service

Atomic approval claim

Capability-scoped execution and authorization recheck

Leasing, dispatch boundary, and recovery

Status and reconciliation API

Atomic invariants

Acceptance criteria

Deployment and live-closure criteria

Test scenarios

Exact preview binding

Concurrent approval

Wrong or stale action

Worker race and stale lease

Crash boundaries

Delivery modes

Authorization change after approval

Reconciliation and manual resolution

Infrastructure and failure delivery

Regression and redaction

Required verification

Infrastructure details

Dependencies and closure ownership

Out of scope

Activity

alexeygrigorev commented on Jul 29, 2026

alexeygrigorev commented on Jul 30, 2026

Software Engineer handoff

Delivered

Changed files

Verification

alexeygrigorev commented on Jul 30, 2026

Blocking findings

Passing evidence

Acceptance criteria, in order

alexeygrigorev commented on Jul 30, 2026

Software Engineer re-handoff — tester findings resolved

Fixes

Final verification

alexeygrigorev commented on Jul 30, 2026

Sole blocker: production transaction concurrency is still simulated

Original counterexamples — now passing

Commands and evidence

Acceptance criteria

alexeygrigorev commented on Jul 30, 2026

Software Engineer re-handoff — real transaction concurrency proof added

Genuine DynamoDB transaction test

Verification

7 remaining items

alexeygrigorev commented on Jul 30, 2026

alexeygrigorev commented on Jul 30, 2026

Product/operator decision

Evidence

Required operator sequence

alexeygrigorev commented on Jul 30, 2026

alexeygrigorev commented on Jul 30, 2026

alexeygrigorev commented on Jul 30, 2026

On-Call update — main 3365c2a

alexeygrigorev commented on Jul 30, 2026

On-Call update — main 98fdc1f

alexeygrigorev commented on Jul 30, 2026

On-Call update — main 4f10a5b

alexeygrigorev commented on Jul 30, 2026

On-Call update — main c47c525

alexeygrigorev commented on Jul 31, 2026

PM STATUS RECONCILIATION — code complete; production acceptance still blocked

alexeygrigorev commented on Jul 31, 2026

PM tracker reconciliation — runtime deployment handoff clarified

alexeygrigorev commented on Aug 13, 2026

PM lifecycle reconciliation — default-off deployed; live closure delegated to #128

Completed evidence

Consolidated live owner and #166 sequence

Exact #124 closure subset

Metadata

Metadata

On-Call update — main `3365c2a`

On-Call update — main `98fdc1f`

On-Call update — main `4f10a5b`

On-Call update — main `c47c525`