You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fix(server): message search keeps results when a short non-Latin word differs in case - #30332
Message search drops a thread when a short word in the query only appears in the message with different case or accents, if that word is not ASCII. Searching ошибка в does not find a message reading В логах ошибка, and fermée éc does not find École fermée à Lyon. The thread also disappears while typing: ошибка ло loses Логи: ошибка until the next letter makes the word long enough for the index.
Call path: the command palette, mobile search and t3_thread_search all reach searchMessages in ThreadSearch.ts. buildMessageSearch sends tokens shorter than three characters as substrings, and messageHitsQuery rejects a row when instr(lower(text), token) = 0. SQLite's lower() folds ASCII only, while messageSearchTokens lowercases with JavaScript, so в never equals В in SQL. messageMatcher, which ignores case and accents, never sees the row.
Change
buildMessageSearch now checks each non-ASCII short token against the case and accent spellings it can match (в as в or В, éc as éc, Éc, ec and so on), with or without the index. Short tokens are at most two characters, so that is at most 16 spellings. The check stays in SQL so these tokens still narrow hits before the hit limit, and messageMatcher still decides the final match. ASCII short tokens stay in SQL, because they are what keep a rare short word from being cut off by the hit limit.
Scope and approval
Small, focused fix of an obvious bug: one filter in one function plus a test. No contract, client or schema change.
Verification
Added ThreadSearch short words outside ASCII > finds them in any case or accent, with or without the index, which runs against real SQLite with the trigram index and again after dropping it.
Before the fix:
FAIL src/orchestration-v2/ThreadSearch.test.ts > ThreadSearch short words outside ASCII > finds them in any case or accent, with or without the index
AssertionError: expected [] to deeply equal [ 'thread:short-russian' ]
Tests 1 failed | 26 skipped (27)
After the fix, the whole file:
Test Files 1 passed (1)
Tests 29 passed (29)
Targeted vp lint and vp fmt --check on apps/server/src/orchestration-v2 and tsc --noEmit in apps/server pass.
Not checked: a running client. This changes which results come back, not how they render.
A second test, narrows short words outside ASCII before the hit limit, puts 2,000 newer messages with only the long word ahead of an older message with both, with and without the index, and fails if the short word stops narrowing before the limit.
Not addressed: an ASCII short word against an accented letter (fermée ec for École) is still rejected in SQL. Letting those through would mean skipping the check for any message with a non-ASCII character, which weakens the narrowing the hit limit depends on. The next letter reaches the index, which folds accents.
… differs in case
A short word after a longer one ("ошибка в") is checked in SQL with
SQLite's lower(), which folds ASCII only, so a message that only has "В"
was dropped before the matcher, which ignores case and accents, saw it.
Non-ASCII short words now skip that SQL check; the matcher still requires
them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Macroscope's review found this PR approvable — Message search now preserves candidates when short non-Latin words differ in case. Final matching, visibility filters, and search limits remain unchanged, keeping the impact limited to search results.
Notes:
Code review disabled. Approvability was decided on eligibility alone.
You can add or adjust custom eligibility rules. Learn more.
No actionable comments were generated in the recent review.
Walkthrough
Thread search now generates spelling alternatives for short non-ASCII query tokens and checks those alternatives in SQL. Tests cover matching across case and accent variations, absent terms, and older matching messages, both with and without the search index.
Changes: Thread search matching
Layer / File(s)
Summary
Short non-ASCII token filtering apps/server/src/orchestration-v2/ThreadSearch.ts
Search requests now carry spelling alternatives for short non-ASCII tokens. Search construction separates those tokens from the existing substring filters, and SQL checks each token against its alternatives.
Search matching and limit regressions apps/server/src/orchestration-v2/ThreadSearch.test.ts
Tests cover Russian and French matches across case or accent variations, absent-token filtering, and an older Russian match despite 2,000 newer partial matches. The tests cover indexed and non-indexed searches.
The remaining narrow accent-search limitation predates this change. No new issue identified here prevents merging after normal checks.
Pre-merge checks: 4
Check name
Status
Explanation
Title check
The title names the main change: message search keeps results when a short non-Latin word differs in case. The objectives and the raw summary confirm the change adds case and accent spellings for shor…
Description check
The description follows the template sections (Problem, Change, Scope and approval, Verification). The Problem section gives the failing inputs, the call path, and the root cause. The Change section e…
Linked Issues check
Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check
Check skipped because no linked issues were found for this pull request.
Full details: Title check
Explanation
The title names the main change: message search keeps results when a short non-Latin word differs in case. The objectives and the raw summary confirm the change adds case and accent spellings for short non-ASCII tokens in the SQL filter. The title is specific and follows the conventional commit format, so it passes.
Full details: Description check
Explanation
The description follows the template sections (Problem, Change, Scope and approval, Verification). The Problem section gives the failing inputs, the call path, and the root cause. The Change section explains the per-token spelling check and why ASCII short tokens stay in SQL. The Scope section gives a justification for a small obvious bug fix without a linked issue, which the template allows. The Verification section lists the new test, its failure before the fix, and passing results after the fix, plus what was not checked. The template asks for the model and harness used, and the description includes that. The description is mostly complete and passes.
Finishing Touches
Generate unit tests (beta)
Create a new PR
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
CodeRabbit Verify
Run CodeRabbit Verify to test the changes in this pull request.
Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 5 remain after this review.
Skipping the SQL check let newer messages with only the long word fill the hit limit before the matcher rejected them. Each short word is now checked as the case and accent spellings it can match, so the filter still runs first.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
verified exact head 6c9fa82fd2c3778da8d7894714ff5796ea80a0b8 on linux using the dev web app, repository playwright, /usr/bin/google-chrome, and real codex gpt-6.1-sol turns through the local proxy. no code changes were needed.
created the project and messages through the app. on github/main (7c22e24e3f), ошибка в and fermée éc returned no threads despite the matching messages. a separate thread disappeared at сбой ло and returned at сбой лог. stopped main's server, then ran this exact pr head against a copy of that dev state.
on the pr, both russian and french searches return the matching thread. the result stays while typing сбой ло to сбой лог. sidebar and command-palette results open the correct thread with find highlighting. ошибка я still rejects the thread. a fresh codex turn completed on the pr, and its Riverbank message was searchable.
main, russian search:
pr, same data and query:
main, french search:
pr, same data and query:
recording of typing retention, result navigation, command-palette search, and absent-word rejection. inspected the clip and verified uploaded playback and seeking:
focused checks ran through blacksmith: vp i && vp test run apps/server/src/orchestration-v2/ThreadSearch.test.ts, all 29 tests passed. these include indexed/fallback searches and filtering before the 2,000-hit limit. the inspected head also had 28 successful ci contexts, zero failing or pending.
native ios/android, the electron shell, and multi-environment/relay/tunnel searches were not exercised on this linux vm. the real app dataset did not contain 2,000 partial matches, and i did not disable its fts index; those paths passed in the focused sqlite tests. the ascii-short-token/accent limitation already documented in the pr remains outside this fix.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
size:S10-29 changed lines (additions + deletions).vouch:unvouchedPR author is not yet trusted in the VOUCHED list.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Message search drops a thread when a short word in the query only appears in the message with different case or accents, if that word is not ASCII. Searching
ошибка вdoes not find a message readingВ логах ошибка, andfermée écdoes not findÉcole fermée à Lyon. The thread also disappears while typing:ошибка лоlosesЛоги: ошибкаuntil the next letter makes the word long enough for the index.Call path: the command palette, mobile search and
t3_thread_searchall reachsearchMessagesinThreadSearch.ts.buildMessageSearchsends tokens shorter than three characters assubstrings, andmessageHitsQueryrejects a row wheninstr(lower(text), token) = 0. SQLite'slower()folds ASCII only, whilemessageSearchTokenslowercases with JavaScript, soвnever equalsВin SQL.messageMatcher, which ignores case and accents, never sees the row.Change
buildMessageSearchnow checks each non-ASCII short token against the case and accent spellings it can match (вasвorВ,écaséc,Éc,ecand so on), with or without the index. Short tokens are at most two characters, so that is at most 16 spellings. The check stays in SQL so these tokens still narrow hits before the hit limit, andmessageMatcherstill decides the final match. ASCII short tokens stay in SQL, because they are what keep a rare short word from being cut off by the hit limit.Scope and approval
Small, focused fix of an obvious bug: one filter in one function plus a test. No contract, client or schema change.
Verification
Added
ThreadSearch short words outside ASCII > finds them in any case or accent, with or without the index, which runs against real SQLite with the trigram index and again after dropping it.Before the fix:
After the fix, the whole file:
Targeted
vp lintandvp fmt --checkonapps/server/src/orchestration-v2andtsc --noEmitinapps/serverpass.Not checked: a running client. This changes which results come back, not how they render.
A second test,
narrows short words outside ASCII before the hit limit, puts 2,000 newer messages with only the long word ahead of an older message with both, with and without the index, and fails if the short word stops narrowing before the limit.Not addressed: an ASCII short word against an accented letter (
fermée ecforÉcole) is still rejected in SQL. Letting those through would mean skipping the check for any message with a non-ASCII character, which weakens the narrowing the hit limit depends on. The next letter reaches the index, which folds accents.Claude Opus 5.5 (Claude Code desktop app)
🤖 Generated with Claude Code