Repository navigation
Atlas step 4: serve mode — job protocol, worker pool, in-memory temp store #5354
Description
Step 4 of the Atlas binary+service epic (#5350). Design of record: doc/2026-11-04T08-07-13Z_design_atlas_binary_service/ (§2 protocol, §3 clients/auth, §6 non-goals).
Scope
atlas-protocrate: the job protocol's request/response types, shared by the server and the CLI's via-service client so they cannot drift.servesubcommand: HTTP+JSON on 127.0.0.1 —POST /v1/search(one-shot),POST /v1/search/jobs(202 + job_id + per-job capability token),GET /v1/search/jobs/{id}(status),GET /v1/search/jobs/{id}/results?cursor=&limit=(pages of the immutable fused ranking),DELETE /v1/search/jobs/{id},GET /v1/search/jobs/{id}/chunk/{chunk_id}(ceiling-checked),GET /health(no auth, no data).- Worker pool: FTS and vector branches of a job execute concurrently; fusion runs once over complete branch pools; fetch-to-fill escalation happens inside the vector branch only.
- Temp store: in-memory, sharded by job_id, TTL 10 min from completion / 60 min hard cap from submit, sweeper-enforced. Nothing persists across restart; in-flight jobs fail
expiredon restart and clients resubmit. (Rationale — reproducible + snippet-bearing + perishable — is design §2; do not re-litigate it into a database without an epic-level decision.) - Auth + ceiling enforcement: bearer tokens from
~/.local/state/atlas-search/service.json(0600), per-clientceiling_cap; effective ceiling = min(requested, cap), recorded on the job and reported in the envelope; theinitverb generates the client table (Drew's CLI + the §3 v1 rows). Aget-chunkoutside the job ceiling returns the missing-chunk shape. - Backpressure (design §2, config-tunable): ≤4 concurrent branch executions globally, ≤2 concurrent + 8 queued jobs per client, pool ≤400 at submit, index-fill escalation ≤3200, saturation → 429 +
Retry-After+ queue depth; queued-too-long jobs failexpired(120 s).
Dependencies: step 2 (core); step 1 via step 3's embedder wiring. Serves the Turso backend only in v1 (local backend stays CLI-side).
Verification gate (exit criteria)
- Protocol conformance tests: submit→page a full ranking and confirm page-concatenation == the one-shot result == the standalone CLI's result for the same query/config.
- Ceiling tests: a
personal-capped token cannot obtain local-only content via search, paging, another job's token, or get-chunk — each attempted path demonstrated closed. - Restart test: in-flight jobs fail
expired, no result data survives on disk anywhere under the state dir. - Saturation test: the 5th concurrent branch execution queues/429s per the configured behavior.
Out of scope: packaging/deploy (step 5), issuing real client tokens for other machines (step 6 cutover), Aster (not a v1 client — design §3/§6).
Acceptance criteria
atlas-protocrate defines the job protocol types shared by server and CLI via-service client;serveexposes the full §2 surface (one-shot search, job submit/status/results paging, chunk fetch, delete, unauthenticated/health) on 127.0.0.1.- Protocol conformance: page-concatenation of a job's results == the one-shot result == the standalone CLI result for the same query/config.
- Ceiling enforcement proven by tests: a
personal-capped token cannot obtain local-only content via search, paging, another job's token, or get-chunk. - Temp store is in-memory only: restart fails in-flight jobs as
expiredand no result data survives on disk under the state dir; TTLs (10 min post-completion / 60 min hard cap) are sweeper-enforced. - Backpressure behaves per design §2: the 5th concurrent branch execution queues/429s with
Retry-After+ queue depth.
Activity
dezren39 commentedon Nov 4, 2026
Intake review — fleet monitor, 2026-11-04.
Related: epic #5350; depends on step 2 (core) and step 1 via step 3's embedder wiring; packaging/deploy is step 5 (#5355), token issuance is step 6 (#5356).
Notes: the in-memory temp store is a decided design point (§2 rationale recorded in the issue) — the acceptance criteria pin the restart/TTL behavior so it is tested, not relitigated. The ceiling tests are the security substance of this step: four distinct exfiltration paths each demonstrated closed.
Risks: protocol drift between server and CLI client is the main structural risk; the shared
atlas-protocrate is the mitigation and is in scope.Suggested approach: proto crate + serve skeleton first, conformance test (three-way equality) early, then ceiling/restart/saturation tests as the exit gate.
Merge pull request #5392 from developing-today/atlas-step4-serve
dezren39 commentedon Nov 4, 2026
Step 4 complete — PR #5392, merged
5acabc82What was built
atlas-proto(new crate): the job protocol's serde types, shared by the server and the CLI's via-service client —SearchRequest(full CLI search surface),JobCreated,JobStatusResponse(incl. queue depth/position),SearchPage,JobDeleted,JobSummary/JobListResponse(recovery),HealthResponse, uniformErrorResponse. No policy lives in proto.crates/atlas-search/src/server.rs, std-only HTTP onTcpListener, loopback enforced):POST /v1/search,POST /v1/search/jobs,GET /v1/search/jobs(recovery list),GET /v1/search/jobs/{id},GET .../results?cursor=&limit=,DELETE .../{id},GET .../chunk/{chunk_id},GET /health(no auth). One shared embedder (the CLI's lazy embedder behind a load-state handle: cold → warming → ready, reported by /health); Turso backend resolved through the existing read-only token chain — the service holds no write credential and builds only SELECTs.BranchPoolExecutorseam (run_search_with_executor); standalonerun_searchuses the default backend executor with byte-identical semantics. Serve mode's executor runs hybrid branches concurrently on scoped threads and fuses once; single-branch jobs execute inline — per Drew's usage profile, the primary real-world path (laptop-local, unfiltered, vector mode on IndexFill/IndexFixed) is a first-class one-slot, one-branch execution with no FTS work and no extra thread hop. Index backends are fully wired through proto, CLI, and server (live-verified below). Core defaults unchanged (the flip stays Atlas step 6: golden-parity gate (Gate B) + cutover #5356's).initverb: generates the v1 client table into~/.local/state/atlas-search/service.json(0600, enforced at load):drew-clicap secret-bearing;openchamber,fork-server,sandbox-scriptscap personal. Token values go to the file only. Serve config also carries listen address, vec_table, and tunable limits.Design delta 1 — async queue (Drew, 2026-11-04)
Submit returns immediately with a job handle in
queued(orrunning). A bounded strict-FIFO execution queue sits in front of the ≤4 concurrent branch executions: per-client caps of 2 running computations + 8 queued, plus a global queue cap of 8; overflow is 429 +Retry-Afterwith the live queue depth in the body. Accepted queued jobs never expire in the queue — the earlier 120 s queue-TTL idea is dropped in favor of bounded admission — and the stored ranking's 10-minute TTL runs from completion (the 60-min hard cap remains only as the pre-completion outer bound). Jobs carryqueue_depth/queue_position. The CLI client polls status, so its visible behavior stays blocking; queueing is a protocol property for agent clients.Design delta 2 — per-client job recovery (Drew, 2026-11-04)
GET /v1/search/jobswith a client bearer token lists that client's own jobs — job id, state, query summary (query/mode/pool), effective ceiling, submit/expiry times, queue position — metadata only, never results, never capability tokens. Job tokens cannot list (401). Pickup after a disconnect is job id + capability token on the existing job endpoints.Design delta 3 — dedupe cache (Drew, 2026-11-04)
Execution is shared through a result cache whose unit is a Computation; every submission still gets its own job id + capability-token handle over the shared immutable ranking (listing/pickup/per-job authorization unchanged).
(query, mode, k, pool, group_by_session, expand, with_message, model, embedder, trunc, vector_backend, kind[], topic[], project[], client[], since, until, effective_ceiling). Exact-key sharing only: the effective ceiling is in the key, so a lower-ceiling request can never be served from a higher-ceiling entry, and filters are in the key, so no stored ranking is ever subsetted to fake a filtered one.donehandle.Verification
cargo test --workspace: 43 passed, 0 failed (1 ignored = Gate A ceremony test, unchanged from step 1).cargo clippy --workspace --all-targets: 0 warnings.cargo fmt --check: clean. Toolchain: rustup 1.99.0,RUSTC_WRAPPER=empty.Retry-After: 7/queue-depth; vector index modes proven single-branch with the rightBranchQueryshapes; recovery list proven (owner sees, other client doesn't, job token can't list, fresh-connection pickup); health/delete.index-fixedandindex-fillunfiltered vector parity, 10 results each; dedupe live (first submit 6 s, identical second 0.00 s, statusdone); recovery list + pickup live; 3 parallel submits →queued/running/running, all complete; /health reports the shared embedder warm. Fallback gate: server killed → CLI prints the loud stderr warning, envelope carries thevia-service: …; served standalonenote, results identical to standalone. Restart gate: pre-kill job id → 404job_expired_or_unknownafter restart; the server wrote zero result-data files anywhere (state is memory-only; the scratch dir held only the config the test created).Notes for step 5 (#5355)
atlas-search initalready producesservice.json(0600) with the v1 table; each client copies only its own token into~/.config/atlas-search/client.json({"name", "token"}, 0600) — the CLI resolves tokens viaATLAS_SEARCH_TOKEN→ATLAS_SEARCH_CLIENT_CONFIG→ that path. Real cross-machine issuance is step 6's, per the epic.ATLAS_ORT_LIBtoday) withlibstdc++.so.6+libz.so.1reachable — the E2E ran with exactly those two store paths onLD_LIBRARY_PATH. Model dir resolution order:ATLAS_MODEL_DIR→~/.local/state/atlas-search/model/<rev>→ HF cache snapshot (Q1 state-dir fetch lands with packaging).servetakes--job-ttl-ms/--hard-ttl-msoverrides for operators/tests; limits otherwise come fromservice.json.