fix(extraction): index Python body docstrings across backends (#1905) - #2004
Merged
Merged
Conversation
#1905) getPrecedingDocstring only walks preceding comment siblings, so a Python docstring — a bare string literal first in the body — never reached the docstring column, never entered nodes_fts, and was never shown by `codegraph node`. The identical sentence written as a leading `#` comment was both. For a Python codebase that is most of the prose there is. (#1905) Adds an optional getBodyDocstring() to LanguageExtractor, in the same shape as getSignature(), and routes every docstring call site through one docstringFor() helper that consults both sources. A node carrying a comment AND a docstring keeps both, joined: they are two things the author wrote about the same symbol and the column is free text. The Python implementation reads the grammar`s string_content rather than slicing quotes off the raw text, so r/u/b prefixes and both triple-quote forms work without a regex per case; f-strings are skipped because an interpolated string is code, not prose. Dedent follows PEP 257 (first line exempt). Scoped to Python deliberately. Julia (a string sibling before the def) and Elixir (@doc) fit the same hook and are left as follow-ups.
Python extraction only consulted preceding comments, leaving body docstrings absent from search and rendered prose. Extend the contributor's hook to module and definition docstrings, and mirror it in the native kernel. Handle comments, concatenated literals, and indentation consistently while rejecting bytes, f-strings, and tuples. Verify persistence, exact-name ranking, MCP/CLI rendering, and native/WASM parity. Co-authored-by: Max Hsu <maxmilian@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Sep 27, 2026
colbymchenry
added a commit
that referenced
this pull request
Sep 27, 2026
* test: close resolution graphs before removing Windows fixtures Close ArkTS and JVM graph connections in afterEach, including failed assertions. Verified both files on Windows 11 and macOS (32 tests). * test: compare explore source lines across LF and CRLF checkouts Normalize only line endings when matching emitted source against checked-out fixtures. Preserve the closure coverage and whole-line assertions. Both files pass on Windows 11 and macOS (15 tests). * test: exercise old Git fallback without a POSIX shell shim Inject only the unsupported ls-files invocation and run every other Git command against real repositories. Assert that each rejected call retries, use portable graph paths, and clean up fixtures. Passes on Windows 11 and macOS. * test: normalize Python fixture before generating CRLF cases The tests added in #2004 produced CRCRLF from a Windows checkout. Normalize the input to LF before constructing either variant. All 28 native and WASM cases pass on Windows 11 and macOS. * test: allow time for eviction fixture indexes on Windows The #2036 eviction test builds and reconciles nine real indexes and consistently exceeds its default five-second timeout on the VM. Allow 15 seconds for this test alone, preserving every assertion. The full file passes on Windows 11 and macOS (21 tests). * test: finish MCP processes before resetting Windows fixtures Launch actual servers with their runtime flags and await termination before directory removal. Use asynchronous removal retries for daemon fixtures. Establish a ready daemon and stop its first proxy before simulating PID reuse, then make source stale without opening a competing SQLite writer. Preserve lock and database byte-equality assertions. All touched files validated on Windows 11 and macOS. * test: allow Windows named-pipe fallback polling to finish The VM takes up to 25.7 seconds to complete the nominal six-second retry loop. Allow 30 seconds for the three fallback waits and 40 seconds for the two shorter enclosing tests, retaining their lock-preservation and read-only assertions. The full daemon file is validated on Windows 11 and macOS. * fix(daemon): retry transient Windows PID-file sharing violations A transient EPERM replacing daemon.pid aborted startup, observed in the quiet-client test and reproduced with a real locked handle at ba3c21e. Retry only Windows sharing errors with five bounded delays, rechecking ownership each time and cleaning temporary files. Keep publication synchronous before client processing. Cover transient/permanent failures, changed ownership, non-retryable errors, and a real Windows handle; require a confirmed daemon round-trip before measuring a quiet session. Validated on Windows 11 and macOS.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Python body docstrings never reached stored prose, so intent-based searches missed documented symbols in the default native backend.
Fix
Build on Max Hsu's PR #1909 and complete function, method, class, and module handling in both backends. Preserve preceding comments, support literal concatenation and normalized indentation, and reject bytes, f-strings, tuples, and later strings.
Validation
../../repro/1905/repro.sh "$PWD": exited 1 before; exited 0 after rebuilding, with both functions searchable.npx tsc && npm run copy-assetsnpm run build:kernelnpx vitest run __tests__/python-body-docstrings.test.ts __tests__/kernel-tsjs-parity.test.ts __tests__/extraction.test.ts __tests__/python-quoted-annotation.test.ts __tests__/python-module-scope-collection-methods.test.ts __tests__/kernel-scaffold.test.ts: 738 passed, 1 conditional skip.npx vitest run __tests__/python-body-docstrings.test.ts: final rerun, 28 passed.npx tsc --noEmitgit diff --checkFixes #1905
🤖 Generated with Claude Code