feat(store): attach document vectors after the write, not before it

[senamakel]

Problem

Memory::store committed a document only after its chunks had been embedded, so every write waited out one embedding round trip. An embedding failure does not fail the write, so a refused or timed-out provider cost its whole latency for nothing.

OpenHuman autosaves the user message and the assistant message after every agent turn, as two sequential store calls, and the turn does not return until both are done. With a provider that refuses (an expired or placeholder key), a traced turn spent 655 ms + 847 ms = 1.5 s there, on the end of every task, and the same two calls cost a round trip each when the provider works.

Change

UnifiedMemory::store_with_taint (the Memory::store path) now commits the row and its chunks vector-less and returns. A background task embeds the chunks and attaches the vectors afterwards.

• New documents_deferred.rs holds the path (documents.rs is already over 1000 lines). It reuses write_document_presanitized, and embed_chunk_texts now delegates to an embedder-taking helper so the task needs no borrow of the store.
• The same secret/PII write gate runs first (write_gate::upsert_document_deferred).
• The attach is one guarded UPDATE per chunk: it matches the chunk id and its text, and only where embedding IS NULL. A rewrite or forget while the embedding is in flight replaces or removes the chunks, so a stale vector finds nothing to attach to.
• A vector-less chunk is the state an embedding failure already leaves, so keyword search is unaffected. Only semantic recall of a just-stored document lags by one provider round trip.

Not changed: upsert_document, upsert_documents (the connector batch path, which wants vectors present when it returns) and put_doc.

Tests

documents_deferred_tests.rs, driven by an embedder whose requests wait on a semaphore:

• returns while the provider is still blocked, with the chunk committed vector-less, then the vector appears;
• a refusing provider leaves the stored document vector-less and the write still succeeds;
• a rewrite in flight does not receive the stale vector (attached count 0, new chunk keeps its own vector);
• the write gate still rejects a secret-looking key;
• Memory::store does not wait on the provider (it would hang on the old code).

cargo test --workspace, cargo clippy --all-targets --all-features -- -D warnings and cargo fmt --check are clean locally.

Follow-up

OpenHuman needs a gitlink bump to pick this up. Separately, the bench harness still runs with a placeholder TinyHumans key, which is what makes those embedding calls fail; that is tracked on the OpenHuman side.

[tinysweeper]

Tiny Sweeper review

⚠️ Review failed for 9eb901059334. the review of #189 did not finish within 900s

[coderabbitai]

Warning

Review limit reached

• Run on-demand review

_{This review includes 6 billable files and costs up to $1.50.}

Or wait 48 minutes for your next included review.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: ce3d9158-ee43-4176-8f3c-0c8e74c35803

📥 Commits

Reviewing files that changed from the base of the PR and between 95ddce3 and 9eb9010.

📒 Files selected for processing (6)

• crates/tinymemory-core/src/store/memory_trait.rs
• crates/tinymemory-core/src/store/namespace_store/documents.rs
• crates/tinymemory-core/src/store/namespace_store/documents_deferred.rs
• crates/tinymemory-core/src/store/namespace_store/documents_deferred_tests.rs
• crates/tinymemory-core/src/store/namespace_store/mod.rs
• crates/tinymemory-core/src/store/write_gate.rs

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.

---

_{Comment @coderabbitai help to get the list of available commands.}