Repository navigation
Windows: get_code_snippet returns "(source not available)" and search_code returns 0 matches for projects indexed under non-ASCII (CJK) paths — indexing itself succeeds #903
Description
Environment
- Windows 10 Pro (10.0.19045), Japanese locale (ANSI code page 932)
- Build:
main-zip-2026-07-06(includes the fix(win): escape spawned command-line args so the index worker gets valid argv #881 worker-spawn quoting fix and the earlier_wfopen/ wide-argv fixes fix: use _wfopen on Windows to support non-ASCII (CJK/U...) file paths #637 / fix(windows): use _wfopen for non-ASCII project paths #703 / fix(win): deliver non-ASCII CLI paths end-to-end (wide argv + CreateProcessW) #891)
Summary
With current main, indexing a repository whose path contains non-ASCII (CJK) characters now works:
index_repository spawns workers cleanly, symbols (Class/Method/Function) are fully extracted, and
the project name encodes non-ASCII bytes as UTF-8 hex.
However, the tools that re-read files from disk at query time still fail for such projects:
| Tool | Result on CJK-path project | Result on same repo via ASCII junction |
|---|---|---|
index_repository (full) |
✅ nodes/edges identical to junction | ✅ |
search_graph / query_graph |
✅ symbols, signatures, complexity all present | ✅ |
get_code_snippet |
"source": "(source not available)" |
✅ full source returned |
search_code |
❌ 0 matches for any pattern | ✅ matches returned |
So extraction-time file reads work, but the query-time read path (snippet source load, grep) apparently
still goes through a narrow-char API and fails on CJK paths.
Repro
mkdir C:\tmp\repro-日本語
@'
def multiply(a, b):
return a * b
class Calculator:
def square(self, x):
return multiply(x, x)
'@ | Set-Content C:\tmp\repro-日本語\util.py -Encoding utf8
codebase-memory-mcp cli index_repository --repo-path "C:/tmp/repro-日本語" --mode fast
# -> status:"indexed", project "C-tmp-repro-e697a5e69cace8aa9e", symbols present:
codebase-memory-mcp cli query_graph --args-file <(echo '{"query":"MATCH (c:Class) RETURN c.name","project":"C-tmp-repro-e697a5e69cace8aa9e"}')
# -> Calculator ... (extraction works)
codebase-memory-mcp cli get_code_snippet --qualified-name Calculator --project C-tmp-repro-e697a5e69cace8aa9e
# -> "source": "(source not available)" <-- BUG (line range + signature are correct)
codebase-memory-mcp cli search_code --pattern multiply --project C-tmp-repro-e697a5e69cace8aa9e
# -> 0 matches <-- BUGCreate an ASCII junction to the same directory (New-Item -ItemType Junction), index via the junction
path: both get_code_snippet source and search_code work, on identical file content.
Verified on a real C# repo as well (56 classes / 280 methods extracted from the CJK path — identical
counts to the junction index — yet snippet source and search_code only work on the junction project).
Expected
get_code_snippet returns the source text and search_code returns matches for projects indexed
under non-ASCII paths, same as for ASCII paths.
Notes / suspected cause
- The stored
file_pathon nodes is absolute and contains the CJK segment
(e.g.C:/Users/.../プロジェクト/.../Foo.cs), and line ranges are correct — only the file-content
read fails. Looks like one or more remainingfopen/narrow-path call sites on the query-time path
(snippet source loader, search_code grep), i.e. the same class of problem previously fixed for other
call sites in fix: use _wfopen on Windows to support non-ASCII (CJK/U...) file paths #637 / fix(windows): use _wfopen for non-ASCII project paths #703 / fix(win): deliver non-ASCII CLI paths end-to-end (wide argv + CreateProcessW) #891. On this machine the ANSI code page is 932, so any
UTF-8 → ANSI narrow conversion of the path cannot round-trip. - Possibly related but distinct open issues: search_code returns 0 matches when the indexed repo path contains a space (macOS) #679 (search_code 0 matches, path with space, macOS),
search_code returns 0 matches for all patterns on macOS arm64 v0.8.1 (graph index healthy) #670 (search_code 0 matches, macOS arm64).
Workaround
Index through an ASCII junction/symlink to the repo; all tools work via the aliased path.
Activity
DeusData commentedon Jul 7, 2026
Thanks for the Windows non-ASCII path retest on current main. Triage: high-priority Windows query/file-read bug under #394.
It is useful that indexing now succeeds and only query-time disk reads fail. The fix should target the file re-open paths used by get_code_snippet and search_code, with a CJK-path regression proving indexed metadata and source retrieval both work.
DeusData commentedon Sep 26, 2026
Thank you, @Rakua-Laqua, for the side-by-side junction comparison! Pinning the failure to query-time reads made the root cause easy to find.
The snippet read itself already used the wide-path
_wfopen, but the containment check in front of it resolved paths with the ANSI_fullpath. On code page 932 that can't round-trip a CJK path, so the check refused the file and you got "(source not available)". Wide-path canonicalisation (#973) replaced it, andsearch_codegot its UTF-8 fix in #956. Both shipped in v0.10.0.Could you rerun your
repro-日本語steps on v0.11.0, without the junction, and confirm that bothget_code_snippetandsearch_codereturn content?github-actions commentedon Sep 26, 2026
Thanks for the report! To move this forward we need a bit more so we can reproduce it ourselves:
codebase-memory-mcp --versionyou're onOnce that's here we'll pick it straight back up. Heads-up: issues left
awaiting-reporterare automatically closed after a few weeks of silence, but a comment reopens the door anytime.github-actions commentedon Oct 18, 2026
This issue has been waiting on more information for 21 days (a version, exact steps, or a public repro), so it's now marked
stale. It will not be closed automatically — a maintainer reviews stale issues by hand. Adding a comment with the details clears the label and puts it straight back in the queue. We're happy to pick it back up the moment we can reproduce it.DeusData commentedon Oct 19, 2026
Thank you again for the CJK-path comparison. Checking back on the September retest request: do both snippet retrieval and text search now work without the ASCII junction on your current version? Please include the version if you can retest; I have no fresh result to claim this resolved on your machine.
DeusData commentedon Oct 25, 2026
Thank you again for the report. If you have a newer CJK-path retest result, please share the version and whether snippet retrieval and search both work without the ASCII junction.