Repository navigation
Allow the usual deploy role to list Sponsor event-source mappings in eu-west-1 #149
Description
Allow the usual deploy role to list Sponsor event-source mappings in eu-west-1
Status: pending — source implementation/review, HUMAN CloudFormation apply, and effective-policy acceptance remain
Tags: bug, infra, testing, P1
Depends on: None
Blocks: #147
Next owner: Software Engineer
Scope and policy decision
Change only the shared DataTalksClub/aws-infra source of truth and its focused tests:
- sandbox/dataops/template.github-actions.yaml
- sandbox/dataops/tests/test_template_github_actions.py
The applied baseline is aws-infra commit d370385556344bb6841f88473c12397c326a6b5a from completed #138. Its DataOps template and tests remain unchanged on the current origin/main line at grooming time. Fetch current origin/main and build from that integration line in a fresh clean isolated worktree; preserve dirty sibling checkouts.
Add exactly this separate statement to SponsorCommunicationsComputeDeployPolicy:
- Sid: SponsorWorkerEventSourceMappingList
Effect: Allow
Action:
- lambda:ListEventSourceMappings
Resource: "*"
Condition:
StringEquals:
aws:RequestedRegion: eu-west-1
AWS Lambda's authorization reference lists no resource type or service-specific condition key for ListEventSourceMappings, so Resource "*" is required and lambda:FunctionArn must not be used. The IAM global aws:RequestedRegion key is always present and restricts the grant to the eu-west-1 endpoint. This separate statement is required so the existing function-conditioned CreateEventSourceMapping and event-source-mapping update/delete/get statements are not weakened.
The #147 runtime still must call ListEventSourceMappings with the exact Sponsor worker function-name and old/new Sponsor stream ARN filters, bounded full pagination, and rejection of duplicates or any unexpected result. This IAM issue does not implement that runtime.
Authoritative references:
- https://docs.aws.amazon.com/service-authorization/latest/reference/list_awslambda.html
- https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-requestedregion
- Allow CI to list active change sets for the exact DataOps stack #138 HUMAN/effective evidence: Allow CI to list active change sets for the exact DataOps stack #138 (comment)
- Allow CI to list active change sets for the exact DataOps stack #138 PM acceptance: Allow CI to list active change sets for the exact DataOps stack #138 (comment)
Acceptance criteria
- Work starts from freshly fetched current aws-infra origin/main containing d370385. Do not branch from, modify, push, merge, cherry-pick, rebase, amend, reset, delete, or apply superseded local commit 8cd8b37ea77c7ad47bd2fb0d10b4c53e17fb94d0 or its preserved branch/worktree.
- The template delta adds exactly the one statement above to SponsorCommunicationsComputeDeployPolicy. lambda:ListEventSourceMappings occurs exactly once in the whole template and nowhere else.
- The new statement contains only the exact Sid, Allow effect, one action, Resource "*", and StringEquals aws:RequestedRegion: eu-west-1. It has no lambda:FunctionArn, ArnLike, IfExists, wildcard/alternate region, additional condition, action, or resource.
- Existing event-source-mapping create/get/update/delete statements, all other permissions, resources, parameters, role properties, managed-policy attachments, and IAM resource inventory remain unchanged.
- Preserve the usual OIDC trust exactly: DataTalksClub/dataops, refs/heads/main, audience sts.amazonaws.com. Add no GitHub Environment, environment subject, alternate repository/ref/audience, StringLike, wildcard subject, secret/variable dependency, role, or assumption path.
- Recompute and freeze the resolved canonical Sponsor compute policy byte count/hash after the change. Enforce the 6,144-character managed-policy quota and the existing stricter ceiling of 5,120, leaving at least 1,024 characters headroom.
- Preserve the d370385 baselines for every unaffected boundary: inline policy 9,551 bytes / 8f686d8da8ea0d4db660f9c6828630f21d1ac2dcff6c3de26fb8556874f1ce59; Sponsor service 1,093 / 30aaa91aa6e00e19e7a04b35c4194075d4e881123e1727093a9001fa4f5dd952; dispatcher 1,510 / d705db32ea1786480f149d63916723b22232dad078f0e7710c0984fb31daf313; observability 5,369 / 435213eed7933be4ae303aae1b14ef72f09a42a4a6cf2c1f6fa99706455b827f; OIDC trust 386 / a4fb561e9b54a2418b750fdc84e0c7ddd7856305fd791d615d40aa85174e68bf; one inline policy, four managed policies attached only to the deploy role, and six attachment slots free.
- Parsed and mutation tests reject a missing/duplicate/moved action or Sid; placement in an existing statement, inline policy, other managed policy, or role; condition removal or operator/key/value/region drift; lambda:FunctionArn or any extra condition; extra action/resource; trust/environment/attachment/parameter drift; and regressions in the existing Repair DataOps OIDC SES configuration-set tag-on-create permission #129/Repair dispatcher generated-role IAM prefix for CloudFormation rollback #130/Allow CI to read the exact processed DataOps stack template #135/Allow CI to list active change sets for the exact DataOps stack #138 and Sponsor negative cases.
- Deterministic policy tests prove ListEventSourceMappings on Resource "*" is allowed only with aws:RequestedRegion=eu-west-1 and is denied for eu-central-1, us-east-1, or missing context. They also prove this new statement does not allow ListFunctions, InvokeFunction, Get/Create/Update/DeleteEventSourceMapping, wildcard Lambda actions, or any non-List permission.
- Run the focused Sponsor/List policy and mutation tests, all sandbox/dataops tests, SAM lint/validation, supported Ruff/source compilation, exact quota/count/fingerprint checks, and git diff --check. No UI changes; screenshots are not required.
- Freeze the accepted base, exact two-path NUL-safe inventory, candidate tree, full-index binary diff hash, template/test hashes, resolved policy fingerprints, quota headroom, and trust/attachment fingerprints. Architecture/Security/IAM, Tester, and PM review the same uncommitted fingerprint.
- After all source gates pass, Software Engineer commits only the accepted two-path candidate with Refs Allow the usual deploy role to list Sponsor event-source mappings in eu-west-1 #149. The integrator merges to current aws-infra main and pushes through the normal local-main flow; no PR and no AWS apply are part of source integration. Keep this issue open with human.
- [HUMAN] A credentialed AWS operator confirms dataops-github-actions is terminal, creates a fresh CAPABILITY_NAMED_IAM change set from the exact merged source, and verifies exactly one non-replacing Modify of SponsorCommunicationsComputeDeployPolicy limited to PolicyDocument. Any role, trust, attachment, managed-policy identity, parameter, tag, or unrelated resource change aborts.
- [HUMAN] Execute only that reviewed change set, require dataops-github-actions UPDATE_COMPLETE, then read back the live managed-policy default version and require the accepted canonical size/hash and exact one-statement delta. Verify unchanged trust, attachments, other policy fingerprints, stack drift/inventory, and no pending shared-stack change set.
- [HUMAN] Reproduce effective custom-policy and deploy-role principal-policy simulations with Resource "*": allow ListEventSourceMappings only for aws:RequestedRegion=eu-west-1; implicit-deny alternate/missing regions and every unlisted action above. Do not weaken OIDC trust or add sts:AssumeRole for testing. As accepted in Allow CI to list active change sets for the exact DataOps stack #138, canonical live readback plus official principal simulation is sufficient effective-policy proof; the first actual call from the usual protected OIDC session belongs to Add a reviewed one-time reset path for the disposable sandbox Sponsor table #147 preflight and must fail closed if reality differs.
- PM accepts the HUMAN source/apply/effective evidence and closes Allow the usual deploy role to list Sponsor event-source mappings in eu-west-1 #149. Add a reviewed one-time reset path for the disposable sandbox Sponsor table #147 remains blocked and must not be dispatched until that acceptance is recorded.
Required verification
From the fresh aws-infra worktree:
uv run --with pytest python -m pytest sandbox/dataops/tests -q
uv run --with pytest python -m pytest sandbox/dataops/tests/test_template_github_actions.py -q -k "Sponsor or EventSourceMappingList"
sam validate --lint --template-file sandbox/dataops/template.github-actions.yaml
uvx ruff check --select E4,E7,E9,F sandbox/dataops/tests/test_template_github_actions.py
python -m py_compile sandbox/dataops/tests/test_template_github_actions.py
git diff --check <accepted-base>
Tester must report exact commands, exit codes/counts, resolved policy matrices, fingerprints, quota headroom, and preservation evidence.
Test scenarios
Exact-region list permission
Given the accepted resolved managed policy and aws:RequestedRegion=eu-west-1
When lambda:ListEventSourceMappings is evaluated on Resource "*"
Then it is allowed by exactly SponsorWorkerEventSourceMappingList and no other statement.
Alternate or absent region
Given eu-central-1, us-east-1, or no requested-region context
When the same action is evaluated
Then it is implicit-denied.
Broader grant or trust drift
Given the action is moved, duplicated, combined with another permission, loses its exact region condition, gains a function condition, or the usual main-branch OIDC trust/environment boundary changes
When parsed, mutation, fingerprint, and simulation tests run
Then the candidate fails.
Source is merged but not applied
Given the accepted aws-infra commit is on main
When #147 readiness is assessed before the credentialed CloudFormation apply and effective-policy acceptance
Then #147 remains blocked because source integration alone changes no live permission.
Effective permission is accepted
Given the reviewed stack update is terminal and live readback/simulations match the accepted candidate
When PM reviews the evidence
Then #149 may close and #147 may resume its normal Software Engineer pipeline; no reset dispatch is authorized by closing #149.
Rollback and out of scope
Rollback uses CloudFormation only: deploy the exact prior accepted template through a fresh reviewed CAPABILITY_NAMED_IAM change set, require terminal success, and re-verify the prior fingerprint. Never patch IAM directly.
Out of scope: editing DataOps or source-system repositories; implementing or dispatching #147; reset, table, stream, mapping, queue, provider, or application mutation; normal app deploy; GitHub Environment creation/configuration; broader Lambda discovery or wildcard actions; OIDC trust changes; direct IAM edits; manual role assumption; and any use or disposal of superseded 8cd8b37.
Grooming authorizes no file edit, commit, push, AWS call, CloudFormation operation, workflow dispatch, or reset.
Activity
[-]Allow the sandbox reset preflight to list Sponsor event-source mappings[/-][+]Allow the usual deploy role to list Sponsor event-source mappings in eu-west-1[/+]alexeygrigorev commentedon Aug 11, 2026
PM GROOMED — least-privilege source and HUMAN activation path are agent-ready
The issue now pins the exact separate one-action Sponsor compute statement, mandatory eu-west-1 RequestedRegion condition, policy/quota/negative tests, current d370385-based integration line, preserved superseded 8cd8b37 boundary, uncommitted Architecture/Security/IAM → Tester → PM source gates, and the later credentialed dataops-github-actions CloudFormation apply plus effective-policy acceptance.
#147 remains blocked until the live permission is PM-accepted. The usual exact DataTalksClub/dataops main-branch OIDC trust remains unchanged, with no GitHub Environment or broader Lambda permission.
No file edit, commit, push, AWS call, stack operation, workflow dispatch, or reset occurred. Exact next owner: Software Engineer, working only in a fresh clean isolated aws-infra worktree from fetched current origin/main.
alexeygrigorev commentedon Aug 11, 2026
TRACKER CORRECTION — HUMAN label deferred until source gates ship
The HUMAN acceptance criteria remain unchanged, but the
humanlabel was premature: Software Engineering, Architecture/Security/IAM review, Tester, PM source acceptance, and source commit/integration/push must complete first.The issue remains agent-ready with exact next owner Software Engineer. Add
humanonly after all agent-verifiable source work has passed and shipped and the credentialed CloudFormation apply/effective-policy gate is the sole remaining work. No scope, acceptance criterion, dependency, file, Git, AWS, workflow, or stack state changed.alexeygrigorev commentedon Aug 11, 2026
ARCHITECTURE PASS — frozen #149 candidate
PASS on base
6d0b2e63c3e5976eb046b61b1de4d29c973e69e9, candidate tree18ca8c27c4e934caf353b7d3ed4be2d8f079a2a6, full-index diff SHA-256a13edf86e35ece2ea66ab7d87acbbc46c5f2faec2a7f627cbf63905eb89b0c1f.Exactly two authorized paths change. Removing the new statement makes the parsed template identical to base. The new Sponsor compute managed-policy statement is the smallest IAM-representable grant: only
lambda:ListEventSourceMappings,Resource: "*", andStringEquals aws:RequestedRegion: eu-west-1. It is in the correct policy attached only to the existing deploy role; trust, parameters, IAM inventory, attachments, and existing permissions are unchanged. Compute policy is 3,893 characters, leaving 1,227 below the stricter 5,120 ceiling and 2,251 below the AWS quota; four attachments remain.This supplies exactly #147s missing list dependency. Runtime function/stream filtering remains in #147 because IAM cannot express those filters. Focused and full tests, SAM lint, Ruff, Python compilation, and diff checks pass.
No blocking finding and no AWS call, edit, commit, apply, push, or dispatch occurred.
alexeygrigorev commentedon Aug 11, 2026
SECURITY PASS — frozen #149 candidate
PASS on base
6d0b2e63c3e5976eb046b61b1de4d29c973e69e9, candidate tree18ca8c27c4e934caf353b7d3ed4be2d8f079a2a6, full-index diff SHA-256a13edf86e35ece2ea66ab7d87acbbc46c5f2faec2a7f627cbf63905eb89b0c1f. No source change is required.The List action appears once, solely in
SponsorCommunicationsComputeDeployPolicy, with exactAllow,Resource: "*", andStringEquals aws:RequestedRegion: eu-west-1. AWS documents no resource-level authorization/service condition for this action, so wildcard resource is necessary. Cross-policy evaluation finds no other matching grant. Existing mapping mutations remain separately resource/function conditioned. Exact main-branch OIDC trust, roles, boundaries, parameters, attachments, and all unaffected fingerprints are unchanged. Region and negative-action matrices fail closed; quotas have ample headroom.This is source-security approval only. #147 AWS dispatch remains blocked until #149 is merged, a HUMAN applies the reviewed
CAPABILITY_NAMED_IAMchange set, the effective policy is read back at the accepted fingerprint, and custom plus principal-policy simulations pass. No AWS call, edit, commit, apply, push, or dispatch occurred.Official semantics: Lambda authorization reference, RequestedRegion, and CloudFormation managed-policy update behavior.
alexeygrigorev commentedon Aug 11, 2026
TEST PASS — frozen #149 candidate
PASS on base
6d0b2e63c3e5976eb046b61b1de4d29c973e69e9, candidate tree18ca8c27c4e934caf353b7d3ed4be2d8f079a2a6, and full-index binary diff SHA-256a13edf86e35ece2ea66ab7d87acbbc46c5f2faec2a7f627cbf63905eb89b0c1f.The NUL-safe inventory contains exactly:
sandbox/dataops/template.github-actions.yamlsandbox/dataops/tests/test_template_github_actions.pyCandidate SHA-256 values are
962835914c20644d4a77b64e58f90bd11367a9a91bc201ce8642ca5e28bbdf26for the template andeade8a778d1bc5c50c78d78dd6e5f096b32acda056e64a88a68e1d619f54fca4for the test file. HEAD, localorigin/main, and remotemainall resolved to the accepted base;d370385is an ancestor and superseded8cd8b37is not.Commands and results:
uv run --with pytest python -m pytest sandbox/dataops/tests/test_template_github_actions.py -q -k "Sponsor or EventSourceMappingList"— exit 0; 25 passed, 49 deselected, 61 subtests passed.uv run --with pytest python -m pytest sandbox/dataops/tests -q— exit 0; 74 passed, 189 subtests passed.sam validate --lint --template-file sandbox/dataops/template.github-actions.yaml— exit 0; valid SAM template.uvx ruff check --select E4,E7,E9,F sandbox/dataops/tests/test_template_github_actions.py— exit 0; all checks passed.python -m py_compile sandbox/dataops/tests/test_template_github_actions.py— exit 0.git diff --check 6d0b2e63c3e5976eb046b61b1de4d29c973e69e9— exit 0.Independent parsed comparison proved that removing only
SponsorWorkerEventSourceMappingListmakes the complete template identical to base.lambda:ListEventSourceMappingsoccurs exactly once and the new statement has only the accepted Sid, Allow effect, single action,Resource: "*", andStringEquals aws:RequestedRegion: eu-west-1.Resolved fingerprints are: compute
3893 / 97dd1ea24436bd32f9a81e394c37281c994dbfd78bfd3f8f66f1c5a55d4d5a4f; inline9551 / 8f686d8da8ea0d4db660f9c6828630f21d1ac2dcff6c3de26fb8556874f1ce59; Sponsor service1093 / 30aaa91aa6e00e19e7a04b35c4194075d4e881123e1727093a9001fa4f5dd952; dispatcher1510 / d705db32ea1786480f149d63916723b22232dad078f0e7710c0984fb31daf313; observability5369 / 435213eed7933be4ae303aae1b14ef72f09a42a4a6cf2c1f6fa99706455b827f; OIDC trust386 / a4fb561e9b54a2418b750fdc84e0c7ddd7856305fd791d615d40aa85174e68bf. Compute headroom is 1,227 below the stricter 5,120 ceiling and 2,251 below 6,144. One inline plus four deploy-role-only managed policies remain, leaving six attachment slots.Policy matrices pass: only
eu-west-1allows ListEventSourceMappings, and only through the new Sid;eu-central-1,us-east-1, and missing region deny. The new statement grants none of ListFunctions, InvokeFunction, Get/Create/Update/DeleteEventSourceMapping,lambda:*, or non-List/non-Lambda actions. Mutation and regression tests reject the required placement, condition, trust, parameter, attachment, and prior-boundary drift. Exact DataTalksClub/dataops main-branch OIDC trust is preserved.No screenshots are required. No AWS call/apply, source edit, commit, push, or dispatch occurred during testing.
HUMAN-only gates still pending:
CAPABILITY_NAMED_IAMchange set containing only the non-replacing managed-policy document modification.UPDATE_COMPLETE, and verify live readback, fingerprints, trust, attachments, drift/inventory, and no pending change set.Source commit/integration and PM acceptance also remain downstream. #147 stays blocked until HUMAN evidence is accepted.
alexeygrigorev commentedon Aug 11, 2026
PM ACCEPTED — frozen source candidate; HUMAN activation remains
ACCEPTED on base
6d0b2e63c3e5976eb046b61b1de4d29c973e69e9, candidate tree18ca8c27c4e934caf353b7d3ed4be2d8f079a2a6, and full-index binary diff SHA-256a13edf86e35ece2ea66ab7d87acbbc46c5f2faec2a7f627cbf63905eb89b0c1f.The exact two-path candidate delivers the smallest intended operator permission: one dedicated
lambda:ListEventSourceMappingsallow onResource: "*", limited byStringEquals aws:RequestedRegion: eu-west-1, without weakening mapping mutations, OIDC trust, attachments, parameters, or any other policy boundary. Architecture, Security, and Tester passed this same fingerprint; Tester recorded the full required commands, 74/74 full-suite tests plus 25 focused tests, mutation/deny matrices, and preserved fingerprints. No UI changed, and no additional public documentation is needed: the issue itself provides the public-safe source, reviewed-change-set, apply, readback, simulation, abort, and rollback handoff.This is source acceptance only. Software Engineer may now commit exactly the accepted two files with
Refs DataTalksClub/dataops#149; the integrator may merge and push through the normal aws-infra local-main flow. Do not useCloses, apply AWS, or close this issue. After source ships, addhumanand leave the issue open for the credentialed operator to review and execute the freshCAPABILITY_NAMED_IAMchange set, requireUPDATE_COMPLETE, verify live compute policy3893 / 97dd1ea24436bd32f9a81e394c37281c994dbfd78bfd3f8f66f1c5a55d4d5a4fplus unchanged trust/attachments/drift/inventory, and reproduce the accepted custom/principal simulation matrix.#147 remains blocked. Source merge does not make the permission effective and authorizes no #147 workflow or reset dispatch; it may resume its normal Software Engineer pipeline only after HUMAN evidence is posted here and PM accepts/closes #149.
alexeygrigorev commentedon Aug 11, 2026
ON-CALL PASS — source CI terminal; HUMAN apply remains separate
Inspected exact aws-infra main commit 680c7c2ddc6e069f72cd5e8d5e638874178dbab7, pushed at 2026-08-11T15:54:41Z.
Source-integrity evidence:
Classification: PASS — expected source-CI no-op because aws-infra has no DataOps source workflow. This does not apply or deploy the template.
Next owner: HUMAN credentialed AWS operator. Create and review the fresh CAPABILITY_NAMED_IAM change set, apply only after the exact one-policy-document delta is confirmed, require UPDATE_COMPLETE, and post live readback plus effective-policy simulation evidence. #147 remains blocked until PM accepts that HUMAN evidence and closes #149.
No AWS, CloudFormation, IAM, workflow-dispatch, source, or checkout mutation was performed by On-Call.
alexeygrigorev commentedon Aug 11, 2026
HUMAN APPLY + EFFECTIVE POLICY PASS — #149 complete
Applied exact aws-infra main commit
680c7c2ddc6e069f72cd5e8d5e638874178dbab7through a fresh reviewedCAPABILITY_NAMED_IAMCloudFormation change set.Pre-execution review showed exactly one non-replacing
ModifyofSponsorCommunicationsComputeDeployPolicy, limited to itsPolicyDocument; the only document addition wasSponsorWorkerEventSourceMappingListwithlambda:ListEventSourceMappings,Resource: "*", andStringEquals aws:RequestedRegion: eu-west-1. No role, trust, attachment, parameter, tag, or unrelated resource changed.Post-apply evidence:
dataops-github-actions:UPDATE_COMPLETEv23893 / 97dd1ea24436bd32f9a81e394c37281c994dbfd78bfd3f8f66f1c5a55d4d5a4flambda:ListEventSourceMappingswithaws:RequestedRegion=eu-west-1=>allowedaws:RequestedRegion=eu-central-1=>implicitDenyThis is the accepted source fingerprint and least-privilege region matrix. The shared-stack blocker is resolved. #147 may proceed through its own fail-closed workflow gates; this comment does not bypass or manually perform the reset.