Skip to content

gate1: kernel + sdk skeletons (bootstrap relayflow output) - #1

Merged
khaliqgant merged 2 commits into
mainfrom
flow/gate1-bootstrap
Aug 27, 2026
Merged

khaliqgant merged 2 commits into
mainfrom
flow/gate1-bootstrap

Conversation

@khaliqgant

Copy link
Copy Markdown
Member

First output of the dogfooded build: the gate-1 bootstrap relayflow (workflows/bootstrap-gate1.yaml) produced this diff end to end.

What's here

  • kernel/ — relayflowd cargo workspace: pure core (simulated clock, no I/O), append-only fsync'd SQLite journal (INSERT-only, fail-closed), memoized resume, cross-process crash_resume test. cargo test 30/30.
  • sdk/ — TS spec types (ladder: deterministic|llm|agent; zero-agent flows legal), YAML→spec compiler (20 malformed-spec rejections), fail-closed JournalClient over unix socket with loopback round-trip test. vitest 41/41.
  • kernel/DESIGN.md — journal entry types incl. Appendix A pins, schema, protocol v0.
  • docs/bootstrap-report.md — honest run report + next three work packages.

Provenance & review integrity

  • Design: claude · kernel: codex · sdk: opencode(glm) · review: claude (after grok — unsupported CLI — and gemini — broken auth — failed; see report).
  • Adversarial review attempt 1 found 6 violations; they were repaired; attempt 2 verified fixes → REVIEW_PASSED. Verdict was then independently spot-checked outside the runner (file sizes, INSERT-only journal, core purity, kill test) — all claims held.
  • Tests re-run independently by the report step: Rust 30/30, TS 41/41.

Engine findings this run surfaced (for the rulebook)

  1. Validator passes unknown CLIs at --dry-run; fails 27 min into the run.
  2. --resume lost the run DB while step outputs existed (best-effort write defect, charter §5) — recovered via --start-from.
  3. Wave-barrier scheduling idled sdk-tests behind the kernel lane.
  4. Runner's repair loop re-prompts a reviewer toward its pass token — verdicts after repair need independent verification (done here).

A human merges.

🤖 Generated with Claude Code

Built by the gate-1 bootstrap relayflow (design: claude / kernel: codex /
sdk: opencode-glm / adversarial review: claude). All verification independent:
- kernel: cargo test 30/30 — append-only fsync'd SQLite journal (INSERT-only),
  memoized resume (results, not code), cross-process crash_resume test,
  pure core (no I/O, no wall clock), completionReason on every completion
- sdk: vitest 41/41 — spec types (deterministic|llm|agent, zero-agent legal),
  YAML->spec compiler with 20 rejection cases, fail-closed JournalClient
  over unix socket with real loopback round-trip test
- adversarial review: 6 violations found and repaired, then REVIEW_PASSED
- report: docs/bootstrap-report.md (honest state + next three packages)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Aug 27, 2026 •

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Free

Run ID: af974587-d94a-4a1d-9db3-9ebe642b46eb

📥 Commits

Reviewing files that changed from the base of the PR and between 367b2eb and b7eab82.

⛔ Files ignored due to path filters (1)
  • kernel/Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (15)
  • docs/RFC-0001-everything-is-a-relayflow.md
  • kernel/Cargo.toml
  • kernel/relayflowd-core/src/machine.rs
  • kernel/relayflowd-core/src/machine/tests.rs
  • kernel/relayflowd-core/src/state.rs
  • kernel/relayflowd/Cargo.toml
  • kernel/relayflowd/src/engine.rs
  • kernel/relayflowd/src/exec_det.rs
  • kernel/relayflowd/tests/crash_resume.rs
  • sdk/src/journal-client.ts
  • sdk/src/protocol.ts
  • sdk/src/validate.ts
  • sdk/tests/journal-client.test.ts
  • sdk/tests/validate.test.ts
  • workflows/bootstrap-gate1.yaml

📝 Walkthrough

Walkthrough

The change adds a Gate 1 workflow platform. It includes Rust workflow semantics, SQLite durability, deterministic execution, a Unix-socket protocol, a TypeScript SDK, shared parity fixtures, crash-resume tests, and bootstrap documentation.

Changes

Gate 1 workflow platform

Layer / File(s) Summary
Specification and compilation boundary
kernel/relayflowd-core/src/entry.rs, kernel/relayflowd-core/src/spec.rs, sdk/src/*, testdata/*
Defines workflow specifications, fail-closed validation, canonical JSON, hashing, kernel conversion, verification gates, budgets, dependencies, and parity fixtures.
Core state machine and verification
kernel/relayflowd-core/src/{clock,journal,machine,retry,state,verify}.rs, kernel/relayflowd-core/src/*/tests.rs
Adds simulated time, scheduling, retries, recovery, memoized outputs, dependency unlocking, budget folding, journal events, and verification results.
SQLite journal and runtime
kernel/relayflowd-journal/src/*, kernel/relayflowd/src/{engine,exec_det}.rs
Adds transactional SQLite journaling, effect deduplication, stream offsets, segment rollover, registry tracking, deterministic command execution, durable resume, and run outcomes.
Protocol, CLI, crash recovery, and gate records
kernel/relayflowd/src/{main,server}.rs, sdk/src/{protocol,journal-client}.ts, kernel/relayflowd/tests/crash_resume.rs, docs/*, workflows/*, .gitignore
Adds CLI and Unix-socket operations, typed SDK requests, fail-closed errors, crash-resume scenarios, test records, Gate 1 documentation, workflow provenance, and ignore rules.

Estimated code review effort: 5 (Critical) | ~120 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant JournalClient
  participant RelayflowdServer
  participant Engine
  participant SqliteJournal
  Client->>JournalClient: connect and send request
  JournalClient->>RelayflowdServer: transmit typed JSON request
  RelayflowdServer->>Engine: dispatch run or journal operation
  Engine->>SqliteJournal: persist or read run state
  SqliteJournal-->>Engine: return journal result
  Engine-->>RelayflowdServer: return outcome
  RelayflowdServer-->>JournalClient: send response or event
  JournalClient-->>Client: resolve request
Loading

Poem

I’m a rabbit with a journal bright
I hop through steps from morn to night
SQLite keeps each trail in line
Retries follow clockwork time
Specs hash neatly, fields align
Gate One records the design

Warning

Some tools did not complete. Review the errors below.

🔧 Clippy (1.97.1)

Clippy execution failed


Note

🎁 Summarized by CodeRabbit Free

Your organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Pro by visiting https://app.coderabbit.ai/settings/billing.

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 367b2eb17f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

role: TypeScript engineer. Implements the authoring SDK and spec compiler.
- name: adversary
cli: grok
cli: claude

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Restore the immutable adversary gate

Revert this change to the workflow that judged the same gate-1 work. The added bootstrap report explicitly says this workflow produced the report and that the adversary CLI was swapped mid-run, while the adversary role is responsible for determining whether the output meets RFC-0001 and AGENTS.md; changing that reviewer during the judged run violates the repository's gate-integrity rail and makes the resulting acceptance evidence untrustworthy.

AGENTS.md reference: AGENTS.md:L29-L30

Useful? React with 👍 / 👎.

Comment thread sdk/src/journal-client.ts
Comment on lines +161 to +163
runStart(spec: FlowSpec): Promise<VerbContract['run.start']['result']> {
return this.request('run.start', { spec });
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Send the kernel dialect from runStart

Convert the FlowSpec with toKernelSpec before sending it, or change this API to require KernelRunSpec. As written, normal output from compileYaml contains authoring fields such as maxIterations, dependsOn, and tagged verification objects, but the server's RunSpec::parse accepts only snake_case kernel fields and flat verification, so valid compiled flows are rejected by the advertised runStart path with invalid_spec.

AGENTS.md reference: AGENTS.md:L14-L15

Useful? React with 👍 / 👎.

Comment thread sdk/src/validate.ts
this.fail(`${at}: expected an object`);
return;
}
const st = step as Record<string, unknown>;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Reject unknown authoring fields

Add allowed-key validation at each authoring object level before normalization. Currently a typo such as depends_on: ["prepare"] passes validateSpec because only dependsOn is inspected, and compileStep then silently discards the misspelled field, allowing the dependent step to run without its intended ordering; root, budget, surface, and permission objects have the same fail-open behavior for unknown keys.

AGENTS.md reference: AGENTS.md:L16-L18

Useful? React with 👍 / 👎.

Comment on lines +60 to +61
let stdout = stdout_reader.join().unwrap_or_default();
let stderr = stderr_reader.join().unwrap_or_default();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Terminate descendants before joining captured pipes

Run deterministic commands in a process group and terminate the whole group on timeout before joining the reader threads. If a command starts a descendant that inherits stdout or stderr, for example sleep 300 & wait, killing only the shell leaves that descendant holding the pipes open, so these joins block until it exits and a configured short timeout_ms can still hang the engine for minutes.

Useful? React with 👍 / 👎.

Comment thread kernel/relayflowd-core/src/machine.rs Outdated
let verified = verification
.as_ref()
.is_some_and(|record| record.verdict == crate::entry::VerificationVerdict::Pass);
let may_retry = attempt < step.max_iterations;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Count semantic iterations separately from crashed attempts

Do not use the raw attempt number to enforce max_iterations after recovery. With max_iterations: 2, if attempt 1 crashes, recovery starts attempt 2, but a verification failure in that replacement reaches attempt < max_iterations == false and exhausts the step after only one semantic result, despite the recovery logic stating that a dead attempt does not consume an iteration; track completed semantic executions separately so crashes do not reduce the configured retry allowance.

AGENTS.md reference: AGENTS.md:L19-L21

Useful? React with 👍 / 👎.

1. sdk journal-client: runStart now converts the authoring FlowSpec to the
   kernel dialect via toKernelSpec before it crosses the wire — the kernel's
   RunSpec::parse is fail-closed and rejects authoring keys (maxIterations,
   dependsOn, tagged verification), so the authoring shape could never start
   a run. RunStartParams is typed as KernelRunSpec; the loopback double now
   mirrors RunSpec::parse strictness and a new test round-trips a spec
   straight from compileYaml through run.start.

2. sdk validate: allowed-key validation at every authoring object level
   (root, step, budget, verification, surfaces + items, permissions).
   Unknown keys are an error naming the nearest valid key ("unknown key
   \"depends_on\" — did you mean \"dependsOn\"?") instead of being silently
   discarded — a typo'd depends_on used to pass validation and lose step
   ordering. Implements RFC covenant 2 (preflight fail-closed) in the
   author's vocabulary per covenant 1; tests pin every level.

3. kernel exec_det: deterministic commands now spawn as their own process
   group leader (CommandExt::process_group) and a timeout SIGKILLs the whole
   group (-pid) before joining the pipe readers — previously only the shell
   died, and a descendant holding the inherited stdout/stderr pipes kept the
   reader joins blocked far past timeout_ms. Test pins that
   `sleep 30 & echo started; wait` with a 250ms timeout returns in
   milliseconds, not 30s. crash_resume's machine-crash kill now also takes
   down the step's (now-detached) process group, preserving its
   no-orphan-finishes-the-effect invariant.

4. kernel machine/state: max_iterations is now enforced against completed
   semantic executions (attempts that produced a verifiable result), tracked
   in StepRuntime::semantic_executions, not the raw attempt number — a
   crashed attempt no longer consumes iteration allowance. completion_actions
   takes the semantic count; recovery_actions permits a replacement while
   semantic executions remain. Core test pins: crash attempt 1, verification-
   fail the replacement, an iteration still remains.

Also: provenance comment in workflows/bootstrap-gate1.yaml documenting the
mid-run adversary change (grok unregistered -> gemini auth dead -> claude,
operator action logged in docs/bootstrap-report.md; no judged agent edited
the gate), and RFC-0001 gains the two covenants that finding 2 implements.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@khaliqgant
khaliqgant merged commit 4631571 into main Aug 27, 2026
1 of 2 checks passed
@khaliqgant
khaliqgant deleted the flow/gate1-bootstrap branch August 27, 2026 12:27
kjgbot pushed a commit that referenced this pull request Sep 1, 2026
…ueue

Long-running bash loop that produces PRs against
`AgentWorkforce/flows` by spawning up to N concurrent Claude Code
agents on `agent-relay`. Each agent picks one task from
`ops/factory/queue.md`, works on it in an isolated scratch
worktree, runs `flows run workflows/preswarm-check.yaml` before
pushing, opens a PR, and returns. The existing
`com.agentworkforce.review-swarm` + `com.agentworkforce.auto-merge`
launchd loops handle review + merge.

WHAT SHIPS (against main, one commit):

    ops/factory/README.md                          113 lines
    ops/factory/brief-template.md                   73 lines
    ops/factory/briefs/hn-monitor-real-cli.md       48 lines
    ops/factory/briefs/preswarm-classifier-test.md  55 lines
    ops/factory/briefs/rulebook-consolidation.md    44 lines
    ops/factory/driver.sh                          247 lines
    ops/factory/queue.md                            39 lines
    ops/factory/spawn-worker.sh                    115 lines

8 files, ~730 lines under `ops/factory/`. No `kernel/` change.
No `sdk/` change.

SCOPE AND RFC-0001 POSTURE

This is scaffolding, not the end state. RFC-0001 §3 gate 3
explicitly targets Factory's ~10 hand-rolled claim protocols for
migration onto the kernel; a bash driver whose queue lives in
markdown and whose claim state is regex-parsed is a NEW hand-rolled
protocol of the shape gate 3 intends to kill. Shipping it now is a
deliberate trade: the concurrent-authoring unblock has to happen
before gate 3 lands (otherwise no one authors the gate-3 PR
either), so this ships with the migration receipt written into it.
README.md §"Scope and RFC-0001 posture" documents the plan; the
follow-up brief is to port the driver to
`workflows/factory-tick.yaml` — one flow run per task,
kernel-owned claim/lease, no markdown mutation — as soon as an
agent-relay-spawn primitive is available in the SDK.

The review-swarm S lens correctly flagged this on iter 1. This
iter accepts the architectural criticism, keeps the bash driver
as scaffolding, and rewrites the README to be honest about it
rather than pretending this is the finished shape.

ITER-1 REVIEW-SWARM BLOCKERS ADDRESSED

M-B1 — false flock claim in README/queue.md. Iter 1 said workers
acquire flock on queue.md; the code has never done that. Only the
driver touches queue.md, and the driver is already single-threaded.
This iter rewrites README §"Concurrency model" and queue.md rules
to describe what actually happens: the DRIVER is single-instance
(guarded by `factory-driver.lock`); its claim rewrites are
sequential; the `factory-queue.lock` is defense-in-depth for a
future sibling script, not a cross-worker primitive. The
`flock() { : ; }` dead-code noop is also removed.

M-B2 — `awk -v body="$BRIEF_BODY"` applies C-string escape
processing. Any `\n`, `\t`, or literal backslash in a brief becomes
something else before the awk program sees it, silently. Briefs
will contain shell snippets, regex, and paths — this bites without
a peep. Fixed in `spawn-worker.sh`: summary and body are
materialized to temp files and the awk program reads them via
`getline`, which consumes bytes verbatim. Escape-preservation
verified locally against a brief containing `\n`, `\t`, a literal
tab, and a sed snippet — round-trip byte-diff came back clean.

S-B1 — hand-rolled claim protocol violates RFC-0001 gate 3
posture. Addressed above under "Scope and RFC-0001 posture".

S-B2 — self-modification rail was advertised fail-closed but
implemented fail-open at spawn (an admitted TODO in the code
comment). Iter 1 relied on the brief text + pre-swarm-check M
lens; both are advisory. This iter adds a DIFF-based enforcement
point in `driver.sh`: after a worker returns `STATUS=opened`, the
driver runs `git diff --name-only origin/main..HEAD` in the
worker's worktree and refuses to record `- [x]` if any file under
`ops/factory/**` appears — the queue line goes to `- [!]` and a
human triages. The spawn-worker comment and README §"Self-modification
rail" now describe the layered advisory-vs-enforcement model
honestly. The diff is the source of truth; the brief text and
pre-swarm-check are the advisory layers.

CONCERNS ADDRESSED

M-C1 (queue `]` footgun): documented explicitly in queue.md rules
with a pointer to the migration plan — the constraint is
irreducible in a markdown-as-queue shim.

M-C2 (unknown `agent-relay fleet spawn` blocking contract) and
M-C3 (zero driver tests): both are limitations; documented in
README §"Known limitations". A `driver-tests` brief is planned
as a follow-up so the state-cycle transitions and crashed-tick
recovery get shell-harness coverage.

S-N1 (git fetch has no timeout), S-N2 (README completeness): the
timeout limitation is now in README §"Known limitations"; a
runaway fetch is recoverable (kill + restart; the driver lock
trap cleans up on most exits).

H (iter 1): the codex process reviewing PR #126 emitted only an
OAuth auth-error dump — the H comment on iter 1 was not a real
verdict. No content changes prompted by it; a swarm re-run should
pick up a real H review this iter.

FAIL-FIRST EVIDENCE (M-B2 fix)

Mutation — revert `spawn-worker.sh` to the iter-1
`awk -v body="$BRIEF_BODY"` shape, then substitute a brief body
containing `\n` and `\t`:

    body='line one\nline two	tabbed'
    printf 'X\n<TASK_BRIEF_BODY>\nY\n' > /tmp/t
    echo "$body" | awk -v body="$body" '/<TASK_BRIEF_BODY>/{print body;next}{print}' /tmp/t

Captured output (verbatim):
    X
    line one
    line two	tabbed
    Y

The literal `\n` became a newline. This is the bug in production:
a shell snippet like `sed -e 's/foo/bar\nqux/'` would have its `\n`
converted to a literal newline before the agent read it, quietly
mangling the instruction.

Restore + new file-based awk approach, same input:
    body="line one\nline two	tabbed"
    ... file-based awk (as in spawn-worker.sh:57-75) ...

Captured output:
    X
    line one\nline two	tabbed
    Y

The `\n` stays as backslash-n bytes. Byte-diff against the input
body: clean.

ITER-2 REVIEW-SWARM BLOCKERS ADDRESSED

M-B1 — `wait "$pid"` cannot see the worker PIDs. The tick body was
`printf … | while read …; do (…) & …; done`, which puts `while`
in a pipe subshell. `&` inside there backgrounds a GRANDCHILD of
the outer script; `$!` inside the subshell names that grandchild;
the outer shell's later `wait "$pid" 2>/dev/null || true` errored
"not a child of this shell" and swallowed the error. The driver
then read a still-empty `$result_file`, classified every worker
as "no FACTORY_RESULT line" → `- [!]`, and called
`release_worktree --force` on a worktree the worker was still
writing in. Every tick both lost the real outcome and yanked the
ground out from under running workers. Fixed by rewriting the
loop as `while read …; do … done < <(printf '%s\n' "$claims")` —
process substitution keeps `while` in the OUTER shell, so `$!`
and `wait` refer to real children of the outer script.

Fail-first demonstration (captured verbatim from a POSIX bash
harness):

    === BEFORE (pipe form) ===
    pid=54433 (in-loop)
    pid=54434 (in-loop)
    outer sees last_pid=
    bash: line 7: wait: `': not a pid or valid job spec
    wait failed exit=1

    === AFTER (process substitution) ===
    pid=54437 (in-loop)
    pid=54438 (in-loop)
    outer sees last_pid=54438
    wait succeeded

In the BEFORE form, the outer shell's `$!` was empty because
`&` never happened in the outer shell. In the AFTER form the
outer shell's `$!` correctly names the last backgrounded child
and `wait` succeeds.

M-B2 — brief-template.md non-negotiable #1 told the agent to run
`git checkout -b factory/<TASK_ID> origin/main`, but
`prepare_worktree` in the driver already ran
`git worktree add -b factory/<TASK_ID> <WORKTREE_PATH> origin/main`
before invoking the agent. An agent following the brief literally
would fail with "branch already exists"; an agent that improvised
would be doing something the brief didn't sanction. Fixed by
rewriting non-negotiable #1 to say the branch and worktree are
ALREADY set up and the agent just needs to `cd` in and start
committing.

Concerns (M-C3–C6) — accepted as follow-ups documented in the
README §"Known limitations"; the state-machine test brief will
land as a separate PR authored by the factory itself once this
lands.

H — iter 2 posted a codex-side auth error dump (no substantive
review), same as iter 1. No content changes prompted by it;
another swarm cycle should pick up a real H verdict now that the
codex worker's OAuth token is back.

S — PASS on iter 2 with the RFC-0001-posture reframing accepted.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@kjgbot kjgbot mentioned this pull request Sep 9, 2026
@kjgbot kjgbot mentioned this pull request Sep 15, 2026
Rahulstark2 pushed a commit to Rahulstark2/flows that referenced this pull request Oct 10, 2026
… .git

Live run AgentWorkforce#1 (Cloud run c5c7bb01) failed at the propose step with EACCES on
the checkout's .git/config: proposeChanges hardened by rewriting it, and
the Cloud sandbox leaves .git unwritable after the agent step, so nothing
was published.

git now runs in a fresh private git dir that borrows the checkout's
objects (read-only, via alternates) and a copy of its index (keeping any
unmerged stages), with the checkout as work tree and cwd. Nothing in the
checkout's .git is written or read as configuration, so an agent-planted
config, hook or info/attributes can never run, which is stronger than
rewriting them. Fixer digest 99ad6a3d.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

Session-Id: 497569fc-550a-44a0-a906-35c233c27ba6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant