Repository navigation
atlas-search: step 3 - CLI mode at Python surface parity (#5353) - #5377
Merged
Merged
Conversation
The atlas-search binary crate: search + get-chunk subcommands over atlas-core/atlas-embed, mirroring the Python CLI flag-for-flag (flags, defaults, output envelope, --text rendering, exit codes). Standalone by default (direct Turso via the read-only token flow, or --backend local against a publish file); the query embedder loads lazily so FTS-only search and get-chunk work with no model present, and vector/hybrid fail loudly without one. --via-service implements the prefer-with-loud-fallback contract now (stderr warning + envelope note, standalone results); the job-protocol client lands behind the same seam in step 4 (#5354). Output is byte-identical to the Python: a CPython-compatible JSON writer (indent=1, repr float formatting) and a line-for-line render_text port, proven by fixture parity — all 12 recorded search cases + both get-chunk cases byte-equal through the binary, plus --text goldens generated by the Python renderer itself.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Caution Review failedThe pull request is closed.
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
This was referenced Nov 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Step 3 of the Atlas binary program (epic #5350), closing #5353: the
atlas-searchbinary crate — the one-shot CLI at Python surface parity, built on atlas-core (step 2, #5372) and atlas-embed (step 1, #5367). Core semantics untouched.What was built
search+get-chunksubcommands (clap) mirroringatlas_search.py's argparse surface: same flags, defaults, output envelope,--textrendering, and exit behavior (die()fatals →atlas-search: <msg>on stderr, exit 2; get-chunk not-found → exit 1; backend/transport failures → exit 1 with the message instead of a traceback).--backend localagainst a publish file / side stores viaLocalBackend. One lazily-loaded embedder per process — FTS-only search and get-chunk never touch the model; vector/hybrid without a model/ORT fail loudly (design §4 degradation parity).json.dumps(indent=1, ensure_ascii=False), CPython float repr — serde_json prints0.000087where CPython prints8.7e-05) and a line-for-linerender_textport.--via-service(Drew Q3): prefer-with-loud-fallback implemented now — probes the service base URL (--service-url/ATLAS_SEARCH_URL/ATLAS_SEARCH_VIA=service, defaulthttp://127.0.0.1:18760), and on any failure prints the warning to stderr and records it in the envelopenotes, then serves standalone. The job-protocol client lands behind this seam in step 4 (Atlas step 4: serve mode — job protocol, worker pool, in-memory temp store #5354); no machine detection in the binary (install-context defaults are Atlas step 5: nix packaging (pkgs/atlas-search) + amd deploy #5355 packaging).Flag parity (Python → Rust)
query/chunk_idpositionals--k,--pool,--mode,--kind,--topic,--project,--client,--since,--until,--max-sensitivity,--group-by-session,--expand,--with-message,--text--backend,--db,--vecdb--model(envATLAS_SEARCH_MODEL),--vec-table(envATLAS_VEC_TABLE)--trunc,--vector-backend--embedderfastembedembedder or a non-Snowflake--modelcannot be served by atlas-embed (Snowflake ST path only) and fails loudly for vector/hybrid, pointing at the Python CLI — never a silent wrong-model embedding. FTS/get-chunk unaffected.--via-service,--service-urlOther declared divergences: help formatting is clap's (flags/defaults identical);
default_db_path's script-relative fallback resolves relative to the binary (only reachable when the canonical path is absent);ATLAS_SEARCH_QVEC_FILEis a test-only hook substituting a fixed query vector — the same monkeypatch the Python fixture generator applies toembed_query.Verification (laptop, rustup 1.99.0, sccache wrapper disabled — the step-1 E0514 trap)
fixtures/core-expectations.jsonrun through the compiled binary against the fixture DB — envelopes equal, and stdout byte-equal to CPython's ownjson.dumpsrendering of the recorded envelopes (14/14).--textgoldens were generated by the Python'srender_textitself and match byte-for-byte. Not-found exit code + stderr pinned by test.ATLAS_ORT_LIB= the v3 venv'slibonnxruntime.so.1.31.1+ gcc-lib/zlib on the loader path): 4 questions, top-10s (chunk ids, scores,rank_sources) identical to the deployed Python CLI on all 4. Cold one-shot latency: Rust 13.0–16.4s vs Python 17.2–25.6s in the same battery (both dominated by model load; Python baseline ~35s colder).--via-serviceagainst the (not yet existing) service → stderratlas-search: warning: --via-service: service at http://127.0.0.1:18760 unreachable (io: Connection refused (os error 111)); falling back to standalone, envelope note recorded, results identical to the standalone run.cargo test --workspace(29 tests incl. 7 new),cargo clippy --workspace --all-targets,cargo fmt --check— all clean.Notes for later steps
service::attempt's probe with the atlas-proto submit/page client; the fallback contract around it (stderr warning + envelope note + standalone) does not change.ATLAS_MODEL_DIR→~/.local/state/atlas-search/model/<rev>→ HF cache snapshot, and ORT fromATLAS_ORT_LIB→~/.local/state/atlas-search/ort/libonnxruntime.so; the packaged build must instead resolve ONNX Runtime from its nix closure, which must carrylibstdc++.so.6+libz.so.1(the dlopen loader-path lesson from opencode#1554 — verified live here with the venv's runtime plus the store gcc-lib/zlib pair).Open in [code]smith · Need help on this PR? Tag
@codesmith-botwith what you need.