Skip to content

Reconcile missing CloudFormation ownership provenance for Sponsor CRM table #140

Closed

Description

@alexeygrigorev

Establish trusted historical ownership provenance for the Sponsor CRM table

Status: done — bounded research accepted; route A is rejected as insufficient and route B is independently owned by #143
Tags: bug, research, infra, data, P1
Depends on: None
Blocks: None — #143 independently blocks #136; #141 remains open/nonblocking and must not be recollected
Next owner: None for #140; the downstream HUMAN AWS Support/remediation owner remains exclusively under #143
Resume condition: Closed. Reconsider route A only through a fresh Architecture and Security review if a newly identified authoritative immutable source explicitly binds the exact historical pair

Lifecycle verification

This bounded research lifecycle is complete. Architecture and Security selected route B, Tester independently verified all ten checked criteria against the public sanitized record without AWS or private-evidence access, and Product Manager accepted the operator outcome after Tester PASS.

#143 owns remediation execution and remains independently blocked on its documented HUMAN Support signal. It is out of scope for #140 and did not need to complete before this research issue closed. #136 remains blocked by #143 and its separate protected-environment gate; closing #140 removes only the completed research-lifecycle gate.

Final research decision

The bounded, hashed #141 investigation found exactly one historical Sponsor CRM table-creation event. That event contains a historical DynamoDB table identifier but does not contain or cryptographically link the exact CloudFormation stack identifier. Authenticated deployment provenance establishes the relevant stack incarnation and logical resource but does not contain the table identifier. No trustworthy artifact binds both exact identifiers.

Architecture and Security therefore classify the historical ownership evidence as insufficient, not conflicting and not unavailable. Matching name/ARN, current logical mapping, current schema/configuration, remaining tags, sequence, and the current table identifier are corroboration only.

Route A is rejected. Route B is selected and fully groomed in #143. #140 has no open evidence or execution dependency and its bounded research lifecycle is accepted. #143 independently owns route-B remediation and blocks #136. #141 remains open only for honest HUMAN session-expiry/revocation and authentic provenance cleanup; it blocks neither #140 nor #143, must not recollect the same evidence, and does not block this route decision.

If AWS Support later supplies an authoritative immutable record that explicitly binds the exact historical pair, route A may be reconsidered only through a fresh Architecture and Security review. It does not reactivate automatically.

Evidence reviewed

Raw events, resource/account/principal identifiers, private storage locations, support correspondence, and operational evidence remain outside this public repository. Public evidence is sanitized only.

Why route A is rejected

A one-time tagless exception required authoritative pre-incident evidence explicitly binding:

  • the exact current CloudFormation stack incarnation; and
  • the exact current DynamoDB table incarnation.

The available source classes split those identifiers across unlinked records. Temporal sequence, matching names, name-derived identifiers, current resource reads, configuration similarity, or absence of observed later mutations cannot create the missing authenticated link. The negative mutation history is useful corroboration but does not prove completeness outside the bounded Event History source.

Any mismatch, ambiguity, incomplete identifier, mutable artifact, unverifiable provenance, or late/current-only evidence fails closed. No code-only bypass is safe.

Selected route B

#143 owns the design and approval path for AWS-supported CloudFormation ownership reconciliation. It requires:

  • AWS Support guidance and private read-only diagnosis;
  • a verified backup/export and retained recovery boundary;
  • an isolated non-production restore and ownership rehearsal;
  • deterministic full key/item integrity plus exact schema/configuration checks;
  • explicit import/replacement/Retain/UpdateReplacePolicy/named-table analysis;
  • downtime, write consistency, load, cost, rollback, cleanup, and hard abort criteria;
  • explicit HUMAN approval for each later mutation; and
  • fresh Architecture, Security, Tester, PM, HUMAN, and On-Call gates.

No production repair is performed under #140. Reaching a route decision does not authorize any action in #143.

No-mutation boundary

Do not under #140:

  • change application, guard, migrator, workflow, IAM, SAM, infrastructure, environment, or provider configuration;
  • create, update, delete, restore, import, replace, or retag the table;
  • manually create reserved aws: tags;
  • create, execute, or delete CloudFormation change sets;
  • dispatch/rerun migrations or deployments, enable flags, seed/smoke production, or touch retained queues;
  • recollect Close out the retained CloudTrail audit session record #141 without a newly identified authoritative exact-pair source and fresh scope; or
  • publish private operational evidence or identifiers.

The normal deployment and Sponsor GSI migration remain prohibited until #143 completes and ownership is independently re-proved.

Acceptance criteria

Test scenarios

Current or split evidence only

Given the stack and table identifiers appear only in separate unlinked sources
When historical ownership sufficiency is assessed
Then route A remains rejected and #143 is the only selected path.

Negative mutation history

Given the bounded source contains no target delete, restore, recreate, or ownership-tag mutation
When it is reviewed
Then it remains corroboration and cannot substitute for the missing exact historical binding.

Support later supplies the exact pair

Given AWS Support later returns authenticated exact-pair provenance
When #143 receives it privately
Then work stops for a fresh #140 Architecture/Security review; route A is not inferred or auto-approved.

#141 procedural closure remains incomplete

Given session expiry/revocation or authentic retrieval provenance is still pending
When route selection is reviewed
Then #141 remains open for HUMAN cleanup but does not trigger recollection and no longer blocks #140's substantive decision.

OIDC boundary remains separate from ownership

The #145/#136 migration uses the existing exact main-branch OIDC trust with no GitHub Actions environment. That identity choice does not establish table ownership and cannot unblock migration by itself. #140 does not alter OIDC, workflow, or ownership state.

Out of scope

  • A generic or one-time tagless exception implementation.
  • Treating current live identity or same-name continuity as historical provenance.
  • Executing Reconcile Sponsor CRM table ownership with CloudFormation #143 remediation.
  • Manual reserved-tag repair.
  • Migration, deployment, environment configuration, feature enablement, production data mutation, or retained-resource cleanup.

Activity

  1. added
    bugSomething is broken
    needs groomingRaw intake that needs PM grooming
    humanCode done or issue blocked on human verification
    infraDeployment and infrastructure
    dataData model, migration, storage
    P1Important
    researchInvestigation before implementation
    and removed
    needs groomingRaw intake that needs PM grooming
    on Jul 31, 2026
  2. alexeygrigorev commented on Jul 31, 2026

    @alexeygrigorev
    MemberAuthor

    PM GROOMED — trusted historical binding or no exception

    Selected the Security decision as the product requirement. Current StackId/logical/physical mapping, name-derived ARN, schema, remaining tags, drift, and a newly observed TableId are corroboration only; they cannot authorize a tagless path.

    #140 is read-only-first. It inventories authenticated pre-incident sources and asks Architecture/Security to decide whether any immutable artifact explicitly binds the exact current StackId to the exact current TableId. If yes, the output is a separately groomed, one-table/one-migration exception issue with strict pinning and removal—not an exception implemented here. If no, the output is a separately reviewed, backed-up HUMAN/AWS ownership-remediation issue. Either way, #136 remains blocked until the selected follow-up actually passes its own gates.

    The missing dataops-v1-production environment remains a separate #136 HUMAN gate. Creating it neither proves ownership nor authorizes migration. No code, AWS, environment, workflow, or data mutation is authorized by this grooming.

  3. alexeygrigorev commented on Jul 31, 2026

    @alexeygrigorev
    MemberAuthor

    ARCHITECTURE DECISION — ROUTE A BLOCK; CURRENT EVIDENCE REQUIRES ROUTE B

    I independently reviewed the complete retained #141 CloudTrail evidence set and compared its binding semantics with the already-reviewed GitHub deployment provenance. I made no AWS call, repository edit, workflow action, or mutation, and this review discloses no raw identifier or private location.

    The sources prove complementary but unlinked facts:

    • the one exact Sponsor CreateTable event is an authenticated pre-incident CloudTrail record and explicitly contains the historical DynamoDB TableId;
    • that event contains no exact CloudFormation StackId, canonical stack-ID tag, CloudFormation stack ARN/path, or other cryptographic link to the stack incarnation;
    • the reviewed GitHub provenance establishes the exact stack deployment/incarnation but contains no historical DynamoDB TableId;
    • timing, same deployment window, CloudFormation service-agent context, table name/ARN/schema, and matching creation sequence cannot bridge that split under Reconcile missing CloudFormation ownership provenance for Sponsor CRM table #140's evidence standard.

    The complete bounded mutation inventory also finds no Sponsor delete, restore, recreate, tag, untag, backup, or update event. That is useful corroboration and shows no conflict in the retained Event History window, but absence of those events does not create the missing exact StackId↔TableId link.

    Accordingly, no safe route A remains from the #141 evidence plus the already-reviewed GitHub provenance. Pinning the two values from separate records would be an inference expressly prohibited by #140, not an attested pair. The correct classification is insufficient, not candidate exact binding and not conflict.

    #141's evidence-content review passed Architecture, but #141 itself still has procedural closure gates: the pre-collection read-only STS identity check must be recorded as a deviation from the literal single-API inventory; the separate phone sandbox role's least-privilege policy and authoritative duration/expiry are not independently proved; explicit retrieval-time/session provenance is absent from the manifest; and revocation/expiry confirmation remains HUMAN. Those gaps do not turn the split records into a binding, but #141 must resolve them before dependency closure.

    ARCHITECTURE VERDICT: BLOCK route A and any tagless exception; select route B after #141 procedural completion and independent Security agreement. The next process artifact must be the separately groomed, backed-up HUMAN/AWS ownership-remediation issue required by #140. #136 remains blocked; no migration, normal deployment, import, replacement, retagging, or other production mutation is authorized by this decision.

  4. alexeygrigorev commented on Jul 31, 2026

    @alexeygrigorev
    MemberAuthor

    SECURITY BLOCK — route A is not proved; route B is required

    I independently reviewed the complete private #141 CloudTrail bundle without making an AWS call.

    The retained CreateTable management event is authentic-looking, complete, and consistently selected; it contains the historical DynamoDB TableId and shows CloudFormation invocation. But it contains no exact CloudFormation StackId, and its request contains none of the CloudFormation ownership tags. The complete bounded Sponsor history has no later tag/delete/restore/recreate event that supplies the missing link. The evidence therefore binds a TableId to a table creation, name/ARN, time, and CloudFormation service invocation—not to the exact current StackId.

    Threat model: the CloudFormation service/role and a deployment time can be shared by multiple stack operations, retries, replacements, or stack incarnations. Names and DynamoDB ARNs are name-derived. Even exact equality between the historical and present TableId would prove table continuity only; it would not create the absent immutable StackId↔TableId edge. Temporal correlation with CI or stack events is not cryptographic or explicit ownership provenance. Accepting it would silently turn the one-time exception into a generic tagless inference policy.

    Security classification:

    • candidate exact binding: no
    • conflict: no trusted conflicting exact pair found
    • unavailable: no for this bounded Event History source
    • insufficient: yes

    Route A is rejected. Do not change the #136 guard, retag the table, migrate GSIs, deploy, or infer ownership from the current mapping.

    The safest next action is route B: the orchestrator/PM should file and fully groom the separate backed-up HUMAN/AWS ownership-remediation issue already required by #140. It must obtain AWS Support guidance, establish verified backup/export and isolated restore rehearsal, define CloudFormation import/replacement/retention handling, integrity and rollback/abort criteria, and require explicit HUMAN approval plus fresh Architecture, Security, Tester, PM, and On-Call gates before any production mutation.

    Do not broaden read access for speculative Config/backup fishing unless an already-retained immutable source is known to explicitly contain both exact identifiers. Name-linked Config records or backup metadata that contains only the TableId would remain corroboration. If AWS Support instead provides a written authoritative exact-pair confirmation before remediation, return that new evidence for a fresh independent route-A review; it is not assumed here.

    #140 and #136 remain blocked until the separately groomed route-B issue completes its required gates. The separate dataops-v1-production environment HUMAN gate also remains unchanged.

  5. alexeygrigorev commented on Jul 31, 2026

    @alexeygrigorev
    MemberAuthor

    PM reconciliation complete. The Architecture/Security verdict is now the operative decision: historical ownership evidence is insufficient, route A is rejected, and route B in #143 is selected. #140 now depends on #143 rather than #141. #141 remains open only for honest HUMAN session/provenance closure, does not block the substantive route, and must not be recollected merely to repair metadata.

  6. removed
    humanCode done or issue blocked on human verification
    on Jul 31, 2026
  7. alexeygrigorev commented on Jul 31, 2026

    @alexeygrigorev
    MemberAuthor

    PM lifecycle reconciliation — research output awaits required Tester/PM gates

    The route-A research decision and Architecture/Security classification remain intact, but specialist verdicts do not replace the mandatory lifecycle. #140 no longer waits for out-of-scope #143 execution: it has no open evidence dependency, now awaits bounded Tester verification of the public sanitized record, and then PM final acceptance. #143 remains the independent downstream remediation blocker for #136. Removed the premature human label and did not close #140 or mark any new criterion complete. No AWS call, private-evidence inspection, recollection, or mutation occurred.

  8. alexeygrigorev commented on Jul 31, 2026

    @alexeygrigorev
    MemberAuthor

    TEST BLOCK — bounded public/sanitized research-output verification

    I independently verified #140 without calling AWS, recollecting CloudTrail, opening private evidence, inspecting raw identifiers, changing repository files, or inferring missing provenance. Architecture/Security specialist verdicts were treated as evidence, not as a substitute for this Tester gate.

    Blocking finding

    The substantive research decision passes, but the current public dependency/owner record is internally contradictory, so acceptance criterion 10 is not yet precise enough for PM acceptance:

    These statements cannot all define one dependency graph. They do not authorize execution, but they make the public handoff ambiguous and fail the checked requirement that owner/action routing be precise.

    Required reconciliation before retest:

    1. In Reconcile missing CloudFormation ownership provenance for Sponsor CRM table #140, replace the stale “Reconcile missing CloudFormation ownership provenance for Sponsor CRM table #140 now depends on Reconcile Sponsor CRM table ownership with CloudFormation #143” sentence with the current truth: Reconcile missing CloudFormation ownership provenance for Sponsor CRM table #140 has no open evidence/execution dependency; after Tester PASS and PM acceptance, Reconcile Sponsor CRM table ownership with CloudFormation #143 independently owns route-B execution and blocks Migrate Sponsor CRM GSIs in a protected stage-only workflow #136.
    2. Amend criterion 10 to distinguish the immediate Reconcile missing CloudFormation ownership provenance for Sponsor CRM table #140 lifecycle owner (Tester, then PM) from the downstream Reconcile Sponsor CRM table ownership with CloudFormation #143 HUMAN owner.
    3. Reconcile Close out the retained CloudTrail audit session record #141 current Blocks explanatory text so it says Close out the retained CloudTrail audit session record #141 blocks neither Reconcile missing CloudFormation ownership provenance for Sponsor CRM table #140 nor Reconcile Sponsor CRM table ownership with CloudFormation #143; Reconcile Sponsor CRM table ownership with CloudFormation #143 separately blocks Migrate Sponsor CRM GSIs in a protected stage-only workflow #136. Historical comments may remain as history, but the current bodies/latest reconciliation must unambiguously supersede them.

    Acceptance criteria, in issue order

    1. PASS — Close out the retained CloudTrail audit session record #141 publicly inventories fixed source classes, access/retention limits, query bounds, pagination, counts, byte sizes, and sanitized SHA-256 metadata.
    2. PASS at authorized public scope — the public inventory explicitly records LookupEvents plus the precollection read-only sts:GetCallerIdentity deviation and claims zero mutation; the unresolved authentic session/provenance gates remain honestly open in Close out the retained CloudTrail audit session record #141. I did not inspect private evidence or infer closure.
    3. PASS at authorized public scope — public Architecture/Security records classify the candidate source classes under the four-state standard.
    4. PASS — current mapping, name/ARN/schema, current TableId, timing, and negative bounded history are explicitly rejected as historical exact-pair proof.
    5. PASS — Architecture and Security independently classify the result insufficient, not conflict/unavailable/exact binding, and reject route A.
    6. PASS — repository-wide issue search found only Reconcile missing CloudFormation ownership provenance for Sponsor CRM table #140 and Reconcile Sponsor CRM table ownership with CloudFormation #143 mentioning tagless/one-time-exception handling; Reconcile Sponsor CRM table ownership with CloudFormation #143 explicitly prohibits that implementation. No exception implementation issue was found or authorized.
    7. PASS — Reconcile Sponsor CRM table ownership with CloudFormation #143 is separately groomed with Support, private baseline, backup/recovery, isolated rehearsal, deterministic integrity, supported CloudFormation-path selection, rollback/abort, cost/downtime, per-phase HUMAN approval, and fresh lifecycle gates. Its criteria remain unchecked and no execution is claimed.
    8. PASS — Migrate Sponsor CRM GSIs in a protected stage-only workflow #136 is open/blocked and explicitly depends on Reconcile missing CloudFormation ownership provenance for Sponsor CRM table #140 Tester/PM completion plus accepted Reconcile Sponsor CRM table ownership with CloudFormation #143 remediation; migration/deployment remain prohibited.
    9. PASS — the high-risk public identifier scan found no AWS ARN/account ID, access-key-shaped token, S3/private filesystem path, or UUID-shaped raw identifier in Reconcile missing CloudFormation ownership provenance for Sponsor CRM table #140 body/comments. The only public URLs are four GitHub issue-comment links. Counts, UTC bounds, modes, and digests are allowed sanitized metadata.
    10. BLOCK — route-B execution correctly stays in Reconcile Sponsor CRM table ownership with CloudFormation #143 and Close out the retained CloudTrail audit session record #141 is nonblocking, but the conflicting dependency and “next owner” statements above make the current public routing imprecise.

    Commands and results

    All read-only commands exited 0 unless noted:

    sed -n 1,220p AGENTS.md
    sed -n 1,760p _docs/PROCESS.md
    sed -n 1,320p .claude/agents/tester.md
    
    gh issue view 140 --repo DataTalksClub/dataops --json number,title,state,author,labels,body,comments
    gh issue view 141 --repo DataTalksClub/dataops --json number,title,state,author,labels,body,comments
    gh issue view 143 --repo DataTalksClub/dataops --json number,title,state,author,labels,body,comments
    gh issue view 136 --repo DataTalksClub/dataops --json number,title,state,labels,body,comments
    
    gh issue list --repo DataTalksClub/dataops --state all --limit 500 --json number,title,body,state --jq <tagless/one-time-exception filter>
    gh issue view 140 --repo DataTalksClub/dataops --json body,comments --jq <combined-public-text> | rg --pcre2 <high-risk-identifier patterns>
    gh issue view 140 --repo DataTalksClub/dataops --json body,comments --jq <combined-public-text> | rg -o <public-URL pattern> | sort -u
    git status --short --branch
    

    The high-risk identifier scan exited 1, meaning no matches. The issue search returned only #140 and #143. The worktree remained main...origin/main with no changes. No code/UI changed, so build, infrastructure mutation, browser tests, and screenshots are not applicable to this bounded research-output gate.

    Retest only after the public routing text is reconciled. Route A remains rejected, #136 remains blocked, #141 must not be recollected, and no #143 action is authorized by this verdict.

  9. alexeygrigorev commented on Jul 31, 2026

    @alexeygrigorev
    MemberAuthor

    PM correction — TEST BLOCK routing ambiguity resolved

    Reconciled the current body to one graph: #140 has Depends on: None and awaits bounded Tester verification, then PM final acceptance. After acceptance, #143 independently owns route-B remediation and blocks #136; #141 blocks neither issue. Replaced the stale dependency sentence and amended criterion 10 to distinguish the immediate #140 lifecycle owner from #143’s downstream HUMAN Support owner. Criterion 10 is left unchecked pending Tester retest. No AWS call, private-evidence access, recollection, repository change, or external action occurred.

  10. alexeygrigorev commented on Jul 31, 2026

    @alexeygrigorev
    MemberAuthor

    TEST PASS — corrected bounded public/sanitized research output

    I independently re-tested #140 after the #140 PM correction and #141 correction. I made no AWS call, recollected no CloudTrail evidence, opened no private artifact, inferred no missing provenance, and changed no repository file.

    The prior routing block is resolved:

    Acceptance criteria

    1. PASS — Close out the retained CloudTrail audit session record #141 publicly inventories the fixed source classes, retention/access limits, query bounds, pagination, counts, byte sizes, and sanitized digests.
    2. PASS at authorized public scope — the public inventory records LookupEvents plus the precollection read-only sts:GetCallerIdentity deviation and zero mutation; unresolved authentic session/provenance gates remain open.
    3. PASS at authorized public scope — public Architecture/Security records classify each candidate source class under the four-state standard.
    4. PASS — current mapping/name/ARN/schema/TableId, timing, and negative bounded history are rejected as historical exact-pair proof.
    5. PASS — Architecture and Security independently classify the evidence insufficient and reject route A.
    6. PASS — the repository-wide issue search found only Reconcile missing CloudFormation ownership provenance for Sponsor CRM table #140 and Reconcile Sponsor CRM table ownership with CloudFormation #143 mentioning tagless/one-time-exception handling; Reconcile Sponsor CRM table ownership with CloudFormation #143 prohibits such an implementation. No exception issue is authorized.
    7. PASS — Reconcile Sponsor CRM table ownership with CloudFormation #143 is separately groomed with Support, backup/recovery, isolated rehearsal, deterministic integrity, supported CloudFormation-path selection, rollback/abort, cost/downtime, per-phase HUMAN approval, and fresh lifecycle gates. All execution criteria remain unchecked.
    8. PASS — Migrate Sponsor CRM GSIs in a protected stage-only workflow #136 remains open/blocked and continues to prohibit migration/deployment.
    9. PASS — the public Reconcile missing CloudFormation ownership provenance for Sponsor CRM table #140 body/comment scan found no AWS ARN/account ID, access-key-shaped token, S3/private filesystem path, or UUID-shaped raw identifier. All observed public URLs use github.com.
    10. PASS — immediate Reconcile missing CloudFormation ownership provenance for Sponsor CRM table #140 ownership is Tester then PM; downstream execution stays under the independent Reconcile Sponsor CRM table ownership with CloudFormation #143 HUMAN owner without treating grooming as mutation authorization.

    Following _docs/PROCESS.md, I changed only criterion 10 from [ ] to [x] after verification. #140 now has 10 checked criteria and no unchecked criteria. No status, label, dependency, or other body content was changed.

    Commands and results

    All commands exited 0 unless noted:

    gh issue view 140 --repo DataTalksClub/dataops --json number,title,state,author,labels,body,comments
    gh issue view 141 --repo DataTalksClub/dataops --json number,state,labels,body,comments
    gh issue view 143 --repo DataTalksClub/dataops --json number,state,labels,body,comments
    gh issue view 136 --repo DataTalksClub/dataops --json number,state,labels,body,comments
    
    gh issue view 140 --repo DataTalksClub/dataops --json body --jq <acceptance-criterion extraction>
    gh issue view 140 --repo DataTalksClub/dataops --json body,comments --jq <combined-public-text> | rg --pcre2 <high-risk-identifier patterns>
    gh issue view 140 --repo DataTalksClub/dataops --json body,comments --jq <combined-public-text> | rg -o <public-URL pattern> | sed -E <host extraction> | sort -u
    gh issue list --repo DataTalksClub/dataops --state all --limit 500 --json number,title,body,state --jq <tagless/one-time-exception filter>
    gh issue view 140 --repo DataTalksClub/dataops --json body | jq <criterion-10-only checkbox update> | gh api --method PATCH repos/DataTalksClub/dataops/issues/140 --input -
    git status --short --branch
    

    The high-risk identifier scan exited 1, meaning no matches. URL inventory returned only github.com. The exception search returned only #140 and #143. The final checkbox read-back reported 10 checked and zero unchecked. The worktree remained clean at main...origin/main.

    No code/UI changed, so build, infrastructure mutation, browser tests, and screenshots are not applicable to this bounded research-output gate.

    READY FOR PM FINAL ACCEPTANCE. This PASS does not authorize AWS Support contact, #143 execution, migration, deployment, ownership mutation, or #141 recollection.

  11. alexeygrigorev commented on Jul 31, 2026

    @alexeygrigorev
    MemberAuthor

    PM FINAL ACCEPTANCE — ACCEPT

    The bounded research outcome is accepted from the operator/safety perspective after Tester PASS.

    PM accepts #140 as fully complete. Status/dependencies/Blocks are reconciled to done / none / none. Close #140 only; do not close #141 or #143, advance #136, or infer any mutation authority.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1ImportantbugSomething is brokendataData model, migration, storageinfraDeployment and infrastructureresearchInvestigation before implementation

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions