[developing-today]

Closes the investigation leg of #6663; the production runtime-alignment call is filed separately (see below).

Root cause (proven by controlled re-run, amd-6, 2026-11-07)

Gate A is a version-matched claim that was run cross-version:

• Fixtures (fixtures/embed-probes.json provenance): recorded 2026-11-04 under ONNX Runtime 1.31.1 (production Python pipeline, atlas venv), intra 8 / inter 2 threads.
• Gate A's recorded PASS (Atlas step 1: embedder parity spike (atlas-embed) — Gate A #5351, PR Atlas step 1: atlas-embed embedder parity spike — Gate A (#5351) #5367): the Rust side loaded libonnxruntime.so.1.31.1 from the atlas venv — version-matched.
• The atlas: Gate A (embedder bit-identity) fails on the live ORT 1.27.1 — ~1e-7 deltas on every probe #6663 failing run: loaded the nix closure's ONNX Runtime 1.27.1 (pkgs/atlas-search/default.nix bakes that store path as the service default since step 5, Atlas step 5: nix packaging (pkgs/atlas-search) + amd deploy #5355).

Controlled re-run this shift, same laptop/CPU/code/fixtures, only the loaded runtime varied:

• ORT 1.31.1: Gate A passes — 0.0 max|d|, 0 bitdiffs, all 11 probes, single and batch.
• ORT 1.27.1: the reported ~1e-7 deltas on ~740–757 of 768 components of every probe.

Eliminated hypotheses: thread counts (the crate defaults intra 8 / inter 2, exactly the provenance values — atlas-embed/src/lib.rs), tokenizer/model drift (token ids exact; model rev + hashes match provenance), CPU kernel change (same CPU passes under 1.31.1; kernel selection differences are a function of the runtime version).

This PR

• gate_a.rs: before loading, compare the fixture provenance onnxruntime_version with the loaded runtime's version — read from the versioned SONAME both layouts carry (the venv lib IS libonnxruntime.so.<x.y.z>; the nix libonnxruntime.so has the versioned sibling). On mismatch the gate panics NOT APPLICABLE naming both versions, instead of printing a parity table for a comparison that cannot hold. Undeterminable version → guard stays silent (never a false failure).
• Non-ignored unit test for the SONAME parser (runs in ordinary CI).
• fixtures/README.md: records the version-matched requirement.

Verified on the laptop: unit test passes; gate passes under 1.31.1; guard fires with the NOT APPLICABLE message under 1.27.1.

Not in this PR

The remaining decision from #6663 — the production corpus was embedded under 1.31.1 while the live service's baked default is the nix 1.27.1 — is a runtime-alignment call (bump the nix runtime, or deliberately re-embed + re-record). Filed as a follow-up issue with the evidence and a recommendation; this PR is compatible with either outcome (a re-record updates the provenance and the guard follows it).

🤖 Lane M queue worker (fallback mode, code#6619)

---

^{Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.}

[coderabbitai]

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: cb2b7351-f192-4764-8520-1c6d267c9144

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• Trigger review

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn