Skip to content

Set up role-agent process for DataOps #1

Closed

Description

@alexeygrigorev

Set up role-agent process for DataOps

Status: pending
Tags: docs, process-docs, P0
Depends on: None
Blocks: #2, #3

Scope

Set up the DataOps issue-driven role-agent process so work in DataTalksClub/dataops can move from raw intake through grooming, implementation, testing, acceptance, merge, and CI/deploy monitoring.

This includes documenting the repo-specific workflow, creating role-agent instructions for Product Manager, Architect, Process Curator, Assistant Engineer, Software Engineer, Tester, and On-Call Engineer, and ensuring GitHub labels match the pipeline described in _docs/PROCESS.md.

The setup should align with _docs/MERGE_PLAN.md Phase 0 and support the first merge-track issues for importing DTC Operations and keeping the docs app running from the merged repo.

Acceptance Criteria

  • _docs/PROCESS.md clearly defines the DataOps issue lifecycle, mandatory gates, role responsibilities, issue format, label taxonomy, merge rules, testing expectations, and human-verification handling.
  • Role-agent instruction files exist for Product Manager, Architect, Process Curator, Assistant Engineer, Software Engineer, Tester, and On-Call Engineer, with repo-specific responsibilities and handoff rules.
  • Role-agent instructions require agents to read _docs/PROCESS.md before issue work and to respect the source-repo boundary for ../dtc-operations, ../datatasks, and ../podcast-assistant.
  • GitHub labels needed by _docs/PROCESS.md exist in DataTalksClub/dataops, including workflow, type, area, and priority labels.
  • Initial merge-track issues are filed or confirmed for Phase 0/Phase 1 work, with raw issues carrying needs grooming until the Product Manager grooms them.
  • A role agent can inspect an issue and determine the next owner, required gates, dependencies, and expected verification steps without relying on chat context.

Test Scenarios

Scenario: Product Manager grooms a raw issue

Given: a raw GitHub issue with the needs grooming label
When: the Product Manager role agent follows the documented process
Then: the issue is rewritten into the groomed format, receives appropriate labels, records dependencies, and no longer has needs grooming

Scenario: Software Engineer starts implementation after grooming

Given: a groomed implementation issue with dependencies and acceptance criteria
When: the Software Engineer role agent starts work
Then: the agent can identify scope, out-of-scope items, required tests, commit expectations, and handoff requirements from the issue and process docs

Scenario: Merge and deploy monitoring follow the pipeline

Given: an implemented issue that passed tester and Product Manager acceptance
When: the orchestrator merges and pushes the approved branch
Then: the On-Call Engineer role agent knows to monitor CI/deploy status and report or fix failures according to the documented process

Out of Scope

  • Importing code or content from ../dtc-operations, ../datatasks, or ../podcast-assistant.
  • Implementing portal, backend, frontend, assistant, or work-engine functionality.
  • Refactoring the source repositories.
  • Replacing local merge workflow with pull requests unless separately requested.

Activity

  1. added
    needs groomingRaw intake that needs PM grooming
    docsDocumentation or process docs work
    portalShared portal shell and UX
    P0Must have
    process-docsSOPs, templates, references, playbooks
    and removed
    needs groomingRaw intake that needs PM grooming
    portalShared portal shell and UX
    on Jun 27, 2026
  2. alexeygrigorev commented on Jun 27, 2026

    @alexeygrigorev
    MemberAuthor

    Grooming complete: issue body now follows the DataOps process format with status, tags, dependencies, scope, acceptance criteria, test scenarios, and out-of-scope items. Removed needs grooming and set process documentation labels.

  3. alexeygrigorev commented on Jun 27, 2026

    @alexeygrigorev
    MemberAuthor

    TEST PASS

    Verified issue #1 against the current repo and GitHub state. No app test suite was run because this change is docs/process/agent-instruction only and does not touch app code, content metadata, routing, frontend behavior, or runtime infrastructure.

    Commands/evidence:

    • sed -n '1,260p' _docs/PROCESS.md and sed -n '261,620p' _docs/PROCESS.md: process doc defines lifecycle, agent workflow, responsibilities, local-only merge rules, mandatory gates, testing expectations, human verification, label taxonomy, temporary-file rules, and DataOps technology/source boundaries.
    • rg -n '^(## Issue Lifecycle|## Agents|## Agent Workflow|## Issue Format|## Merging|## Mandatory Steps|## Testing Expectations|## Human Verification|## Labels)' _docs/PROCESS.md: required process sections found, including new ## Issue Format.
    • printf '%s\n' product-manager designer architect process-curator assistant-engineer software-engineer tester oncall-engineer | while read r; do test -f ".claude/agents/$r.md" ...: all required role-agent files are present.
    • git ls-files --others --exclude-standard .claude/agents: confirms the eight new agent files under .claude/agents/.
    • rg -n 'Read _docs/PROCESS.md|Respect source boundaries|\.\./dtc-operations|\.\./datatasks|\.\./podcast-assistant|DataTalksClub/dataops|Do not commit|Do not edit code|PM acceptance|Tester|On-Call|Handoff' .claude/agents: role files are DataOps-specific, require reading _docs/PROCESS.md, include handoff rules, and preserve source repo boundaries.
    • gh label list --repo DataTalksClub/dataops --limit 200 --json name --jq '[.[].name] as $labels | ["needs grooming","enhancement","bug","docs","migration","research","portal","process-docs","work-engine","assistant","podcast","frontend","backend","infra","testing","data","design","P0","P1","P2","human"] | map(select($labels | index(.) | not))': returned []; required workflow/type/area/priority/special labels exist, including design.
    • gh issue list --repo DataTalksClub/dataops --state all --limit 100 --json number,title,state,labels ...: initial merge-track issues are present, including Import DTC Operations into dataops #2, Keep DTC Operations docs app running from merged repo #3, Import DataTasks into work-engine #4, Import Podcast Assistant into assistants/podcast #7, Define import log and source commit policy #12, and later Phase 0/Phase 1 follow-ups; raw backlog items retain needs grooming.
    • gh issue view 2, gh issue view 3, gh issue view 4, gh issue view 7, gh issue view 12: sampled merge-track issues expose status, tags, dependencies, blockers, scope, acceptance criteria, test scenarios, out-of-scope boundaries, and verification commands or expectations, so a role agent can determine next owner/gates/dependencies/verification from issue/process docs.
    • git diff --check: passed with no whitespace errors.

    Acceptance criteria:

    • PASS: _docs/PROCESS.md defines lifecycle, mandatory gates, role responsibilities, issue format, label taxonomy, merge rules, testing expectations, and human verification.
    • PASS: role-agent files exist for product-manager, designer, architect, process-curator, assistant-engineer, software-engineer, tester, and oncall-engineer.
    • PASS: role instructions require reading _docs/PROCESS.md and respecting ../dtc-operations, ../datatasks, and ../podcast-assistant boundaries.
    • PASS: required GitHub labels exist, including design.
    • PASS: initial merge-track issues exist/are confirmed; raw backlog items carrying needs grooming are visible.
    • PASS: sampled issues plus process docs provide owner, gates, dependencies, and verification without chat context.
  4. alexeygrigorev commented on Jun 27, 2026

    @alexeygrigorev
    MemberAuthor

    ACCEPTED

    PM acceptance after Tester pass: the role-agent process is usable for DataOps V1 orchestration.

    Reasoning:

    • _docs/PROCESS.md now defines the issue lifecycle, mandatory gates, role ownership, issue format, labels, merge rules, testing expectations, and human-verification handling clearly enough for role agents and the orchestrator to operate from GitHub issues.
    • The eight .claude/agents/*.md role files cover PM, Designer, Architect, Process Curator, Assistant Engineer, Software Engineer, Tester, and On-Call, with repo-specific preflight, source-repo boundaries, and handoff rules.
    • Required labels exist in GitHub, including design, and the current backlog has groomed merge-track issues plus raw needs grooming items in the expected state.
    • Sampled Phase 0/Phase 1 issues expose next owner, dependencies, acceptance criteria, verification, and out-of-scope boundaries without relying on chat context.

    No follow-up required for #1 before commit/merge.

  5. alexeygrigorev commented on Jun 27, 2026

    @alexeygrigorev
    MemberAuthor

    On-call post-push verification for 57aa9aa1adae6dee3e16be939c250f433bd05964 (Add DataOps role agent instructions): PASS / expected CI skip.

    • origin/main points to 57aa9aa1adae6dee3e16be939c250f433bd05964.
    • Changed paths: _docs/PROCESS.md, .claude/agents/*.md.
    • Validate DataOps Content push filter only includes content/** and .github/workflows/validate-dataops-content.yml; no match.
    • Deploy DataOps V1 Lambda push filter only includes workflow/content/frontend/lambda/scripts/tests/work-engine/package/pyproject/uv files; no match.
    • GitHub Actions check: no runs/check-runs for this commit, which matches the path-filter expected skip. No on-call fix needed.

    Run IDs: none for this SHA; expected skip by path filters.

  6. added a commit that references this issue on Aug 14, 2026
    4609d39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P0Must havedocsDocumentation or process docs workprocess-docsSOPs, templates, references, playbooks

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions