Skip to content

Python: module-attribute callers still missed in large package tree after #715 / 1.0.1 #899

New issue
New issue
Closed
#927
Closed
Python: module-attribute callers still missed in large package tree after #715 / 1.0.1#899
#927

Description

@JosefAschauer
JosefAschauer
opened on Jun 15, 2026

Summary

Issue #578 / PR #715 appears fixed for the clean minimal Python module-attribute cases in codegraph@1.0.1, but the original RAGFlow production case still has missing calls edges after a full reindex.

The narrowed finding:

  • from pkg import mod; mod.target(...) now works in a fresh scratch project.
  • import pkg.mod as mod; mod.target(...) now works in a fresh scratch project.
  • from api.db.services import dialog_service; dialog_service._private_shim(...) also works in a fresh scratch project.
  • But in the real RAGFlow tree, from api.db.services import dialog_service; dialog_service._repair_grounded_citation_alignment(...) still reports no callers, even though both caller and callee nodes are indexed and the call expressions are visible in codegraph node.

So the broad #578 bug is fixed, but there is still a narrower miss in the larger package/import context.

Environment

  • codegraph --version -> 1.0.1
  • Linux
  • Project: RAGFlow fork
  • Full reindex performed after upgrading:
codegraph index .

The index reported:

Indexed 2,982 files
0 nodes, 4,425 edges in 14.9s

Control checks that now pass

I created a fresh scratch project under /tmp/codegraph-attr-test:

# pkg/mod.py
def target(value: int) -> int:
    return value + 1

# use_from_pkg_import_module.py
from pkg import mod

def caller_from_pkg_import_module() -> int:
    return mod.target(1)

# use_import_pkg_module_as_module.py
import pkg.mod as mod

def caller_import_pkg_module_as_module() -> int:
    return mod.target(2)

# use_from_pkg_mod_import_target.py
from pkg.mod import target

def caller_direct_import() -> int:
    return target(3)

After codegraph init /tmp/codegraph-attr-test:

codegraph callers target -p /tmp/codegraph-attr-test

correctly reports all three callers:

caller_from_pkg_import_module
caller_direct_import
caller_import_pkg_module_as_module

I also tested the exact package-level import shape in a fresh scratch project under /tmp/codegraph-dialog-shape:

# api/db/services/dialog_service.py
def shim(value: int) -> int:
    return value + 10

def _private_shim(value: int) -> int:
    return value + 20

# test/unit_test/api/db/services/test_prompt.py
from api.db.services import dialog_service

def test_calls_module_attribute() -> None:
    assert dialog_service.shim(1) == 11

def test_calls_private_module_attribute() -> None:
    assert dialog_service._private_shim(1) == 21

After codegraph init /tmp/codegraph-dialog-shape:

codegraph callers shim -p /tmp/codegraph-dialog-shape
codegraph callers _private_shim -p /tmp/codegraph-dialog-shape

both correctly report the test callers.

Real RAGFlow case that still fails

The real code has:

# test/unit_test/api/db/services/test_dialog_service_prompt_sanitization.py:21
from api.db.services import dialog_service

and calls such as:

# test/unit_test/api/db/services/test_dialog_service_prompt_sanitization.py:1770
answer, cited = dialog_service._repair_grounded_citation_alignment(
    "Feng Shi Pian and Feng Shi Xiao Tong Pian are listed as formulas [ID:2].",
    ledger,
)

# test/unit_test/api/db/services/test_dialog_service_prompt_sanitization.py:1774
answer = dialog_service._preserve_grounded_reference_labels(answer, ledger)

The target functions are indexed:

codegraph node _repair_grounded_citation_alignment -p /home/jo/rf/ragflow_p1
codegraph node _preserve_grounded_reference_labels -p /home/jo/rf/ragflow_p1

and the caller function is indexed with the call expressions visible:

codegraph node test_repair_grounded_citation_alignment_replaces_wrong_valid_id_before_label_injection -p /home/jo/rf/ragflow_p1

shows:

answer, cited = dialog_service._repair_grounded_citation_alignment(...)
answer = dialog_service._preserve_grounded_reference_labels(answer, ledger)

But callers are still missing:

codegraph callers _repair_grounded_citation_alignment -p /home/jo/rf/ragflow_p1
codegraph callers _preserve_grounded_reference_labels -p /home/jo/rf/ragflow_p1

Observed:

No callers found for "_repair_grounded_citation_alignment"
No callers found for "_preserve_grounded_reference_labels"

Same symptom for other module-attribute calls from the same test file, for example:

dialog_service._compute_prior_weight(...)
dialog_service._expand_query_with_symbols(...)

codegraph callers _compute_prior_weight reports internal/import-related callers, but not the test functions that call it via dialog_service._compute_prior_weight(...).

Expected

After #715, the real RAGFlow production case from #578 should also get calls edges:

test_repair_grounded_citation_alignment_replaces_wrong_valid_id_before_label_injection
  -> _repair_grounded_citation_alignment

test_repair_grounded_citation_alignment_replaces_wrong_valid_id_before_label_injection
  -> _preserve_grounded_reference_labels

and similarly for the other dialog_service.<function>(...) test calls.

Why this matters

The practical failure mode is still the same as #578: these functions look unused through codegraph_callers, even though regression tests call them by module-attribute access. That makes dead-code cleanup and impact analysis unsafe unless users pair CodeGraph with textual grep.

The fact that fresh scratch projects pass suggests the remaining bug is not the original broad resolver gap, but something about the larger package/import context in the real RAGFlow tree.

Ask

Can you take a second look at Python module-attribute resolution for larger package trees where:

from api.db.services import dialog_service
dialog_service.some_function(...)

The minimal fixture now passes in 1.0.1, but the original production case still misses the edge after a full reindex.

Activity

  1. maxmilian commented on Jun 18, 2026

    @maxmilian
    maxmilian
    on Jun 18, 2026
    Contributor

    Thanks for the unusually precise narrowing — the scratch-vs-real split is exactly the right way to corner this.

    I built codegraph@main and tried to reproduce the remaining miss with isolated resolution tests that match your documented shape and then some, to find the trigger. All of these pass (the calls edge resolves correctly), so they rule those factors out as the cause on their own:

    • nested package path from api.db.services import dialog_service + __init__.py at every level
    • the test living in a deep test/unit_test/api/db/services/… dir
    • a leading-underscore target (_repair_grounded_citation_alignment)
    • a multi-line call expression (args on separate lines, exactly like lines 1770–1773)
    • a @functools.lru_cache(...)-decorated target
    • the caller being a unittest.TestCase class method rather than a bare def
    • a same-name collision — a decoy agent/services/dialog_service.py that also defines _repair_grounded_citation_alignment (resolution still picks the correct api/db/services one)

    So in isolation none of these reproduce it, which lines up with your read that it's something about the larger tree.

    Code-level, a missing edge here means resolvePythonModuleMember (src/resolution/import-resolver.ts) returned null, which happens for exactly one of three reasons:

    1. the test file's extracted imports don't include a binding with localName === "dialog_service", or
    2. findPythonModuleFile("api.db.services.dialog_service", …) matched no file ending in api/db/services/dialog_service.py, or
    3. the member was found-but-rejected — it only accepts kind function/class/variable/constant and deliberately excludes method (so a mod.foo never lands on a same-named class method).

    To pin which branch it is in the real tree, could you grab a few signals from ragflow_p1:

    • codegraph node _repair_grounded_citation_alignment — what kind does it report (function vs method), and is it defined at module level or nested inside a class / if / try?
    • Is there exactly one file whose path ends api/db/services/dialog_service.py, or more than one dialog_service.py in the tree?
    • The exact import line in the test file — is it literally from api.db.services import dialog_service, or multi-name / aliased / inside a TYPE_CHECKING block?
    • Your reindex logged 0 nodes, 4,425 edges — was that an incremental codegraph index . over an already-populated DB? A clean codegraph init (fresh DB) vs an incremental re-index can resolve differently; worth confirming the miss survives a from-scratch index.

    If any one of those points at the failing branch, a minimal fixture (and a fix) follows quickly. Happy to keep digging once we know which of the three it is.

  2. JosefAschauer commented on Jun 18, 2026

    @JosefAschauer
    JosefAschauer
    on Jun 18, 2026
    ContributorAuthor

    Thanks, I collected those signals and also ran the clean-index check.

    Signals from the real ragflow_p1 tree:

    • codegraph node _repair_grounded_citation_alignment -p /home/jo/rf/ragflow_p1 reports kind=function, module-level, at api/db/services/dialog_service.py:784. It is not nested in a class/if/try.
    • There is exactly one file ending in api/db/services/dialog_service.py.
    • The import line is a literal top-level import: test/unit_test/api/db/services/test_dialog_service_prompt_sanitization.py:21 has from api.db.services import dialog_service. It is not aliased, multi-name, or inside TYPE_CHECKING.
    • The existing real repo index still missed the callers: codegraph callers _repair_grounded_citation_alignment -p /home/jo/rf/ragflow_p1 returned no callers.

    Then I tested a fresh DB without touching the real repo index:

    1. Copied the repo to /tmp/ragflow-p1-codegraph-clean, excluding .git, .codegraph, .venv, web/node_modules, generated dirs, logs, and large resource dirs.
    2. Ran codegraph init /tmp/ragflow-p1-codegraph-clean.
    3. The clean index reported Indexed 3,291 files, 67,456 nodes, 208,154 edges.
    4. On that clean index, the callers resolve correctly.

    Fresh-index results:

    codegraph callers _repair_grounded_citation_alignment -p /tmp/ragflow-p1-codegraph-clean

Callers of "_repair_grounded_citation_alignment" (4):
- test_repair_grounded_citation_alignment_replaces_wrong_valid_id_before_label_injection
- test_repair_grounded_citation_alignment_keeps_supported_valid_id
- test_repair_grounded_citation_alignment_replaces_wrong_text_evidence_id
- test_repair_grounded_citation_alignment_preserves_citations_missing_from_partial_ledger

    Also:

    codegraph callers _preserve_grounded_reference_labels -p /tmp/ragflow-p1-codegraph-clean

    returns the expected 5 test callers, and:

    codegraph callers _compute_prior_weight -p /tmp/ragflow-p1-codegraph-clean

    returns async_chat plus the expected test callers.

    So 1.0.1 / current resolver behavior appears to fix the module-attribute caller issue on a clean index. The remaining repro is: an existing .codegraph DB that had previously missed these edges did not recover after codegraph index . incremental reindex. My best read is that this issue is now about stale/incomplete incremental index invalidation, not resolvePythonModuleMember failing on this import shape anymore.

  3. JosefAschauer commented on Jun 18, 2026

    @JosefAschauer
    JosefAschauer
    on Jun 18, 2026
    ContributorAuthor

    One more useful signal: I hit a very small incremental-index repro in the real tree.

    Sequence:

    1. Moved the stale real .codegraph aside.
    2. Ran codegraph init . in /home/jo/rf/ragflow_p1.
    3. Clean init output: Indexed 3,291 files, 67,456 nodes, 208,154 edges.
    4. Verified callers resolved:
      • _repair_grounded_citation_alignment returned 4 test callers.
      • _preserve_grounded_reference_labels returned 5 test callers.
    5. Edited only the docstring text in api/db/services/dialog_service.py around the wrapper comment. No call sites changed.
    6. Ran codegraph index ..
    7. Incremental output: Indexed 3,291 files, 0 nodes, 5,296 edges.
    8. Immediately after that incremental reindex, both caller queries regressed to no callers:
    codegraph callers _repair_grounded_citation_alignment -p /home/jo/rf/ragflow_p1
# No callers found

codegraph callers _preserve_grounded_reference_labels -p /home/jo/rf/ragflow_p1
# No callers found

    Then I moved that broken incremental DB aside and ran another clean codegraph init .; the callers resolved again.

    So the repro is now tighter: a clean index resolves module-attribute callers correctly, but an incremental codegraph index . after a docstring-only edit in the target module can drop those caller edges. I preserved the broken incremental DB locally as /tmp/ragflow-p1-codegraph-incremental-broken-20260618 in case there is a useful local inspection command to run against it.

  4. JosefAschauer commented on Jun 18, 2026

    @JosefAschauer
    JosefAschauer
    on Jun 18, 2026
    ContributorAuthor

    @/tmp/issue899-followup-corrected.md

  5. JosefAschauer commented on Jun 19, 2026

    @JosefAschauer
    JosefAschauer
    on Jun 19, 2026
    ContributorAuthor

    Follow-up 2: #894 already fixes the index path on main; sync still broken

    Built main (commit f34f606, post-#894, not yet released — v1.0.1 is still the latest tag) and re-ran the same docstring-edit repro against it. Results:

    Operation (after docstring-only edit to callee file) v1.0.1 (installed) main (post-#894)
    codegraph index . ✗ 0 + 0 (reported 0 nodes, 5,295 edges) ✓ 4 + 5 (reported 67,445 nodes, 211,532 edges)
    codegraph sync . ✗ 0 + 0 ✗ 0 + 0 (still broken)
    File-watcher auto-sync (calls sync()) ✗ 0 + 0 ✗ 0 + 0 (still broken)

    So #894 (make index a clear-then-rebuild) already closes the index path — once it ships, codegraph index . becomes a safe refresh. Thanks for that.

    The remaining bug is sync() only — which is the path the watcher uses, so every codegraph serve --mcp editor/agent integration is affected. sync()'s filesystem reconcile re-indexes just the changed callee file via indexFile (extraction/index.js:1611 on main) and, in doing so, drops the incoming calls edges from unchanged caller files that import it via module-attribute access. A docstring-only edit on the callee is sufficient to trigger it.

    I'm going to look at a fix for the sync path — likely preserving existing cross-file calls edges whose caller endpoint still has the call expression in its indexed body, rather than dropping them on callee re-index. Will open a PR if it pans out.

  6. JosefAschauer commented on Jun 19, 2026

    @JosefAschauer
    JosefAschauer
    on Jun 19, 2026
    ContributorAuthor

    Opened PR #927 with a fix for the sync path: snapshot incoming cross-file edges before the callee re-index, re-resolve targets by (filePath, kind, name) after, re-insert. Two regression tests model the RAGFlow repro (line-shift case + rename case). Full suite green (1412 passed). Closes #899.

  7. colbymchenry
    added a commit that references this issue on Jun 21, 2026

    fix(sync): preserve cross-file caller edges across callee re-index (#899

    6110df8
  8. colbymchenry
    closed this as completedin #927on Jun 21, 2026

  9. colbymchenry commented on Jun 21, 2026

    @colbymchenry
    colbymchenry
    on Jun 21, 2026
    Owner

    Resolved — and thank you for the outstanding diagnostic work here, @JosefAschauer. The scratch-vs-real narrowing that proved a clean index resolves fine, then pinning the regression to the incremental path, was what cracked this.

    For posterity, the two-part resolution:

    I reproduced the RAGFlow shape, confirmed #927 fixes it, and the full suite is green. The fix is language-agnostic, so it also covers the same edge-drop for other languages (and even synthesized cross-file edges that sync was dropping). Landing in the next release.

    Thanks again for the careful narrowing and the clean PR with regression tests. 🙏

  10. JosefAschauer commented on Jun 21, 2026

    @JosefAschauer
    JosefAschauer
    on Jun 21, 2026 via email
    ContributorAuthor
  11. colbymchenry
    added a commit that references this issue on Jun 21, 2026

    docs(changelog): note the sync cross-file caller-edge fix (#899)

    212dfc4
  12. colbymchenry
    added a commit that references this issue on Jul 10, 2026

    fix(sync): resolve cross-file refs when an edit adds or removes the s…

    9d0cd3a

  13. br41nlet commented on Jul 25, 2026

    @br41nlet
    br41nlet
    on Jul 25, 2026

    Reproduced on 1.5.0, and the remaining cause looks like an alias mismatch rather than a package-depth problem.

    resolvePythonModuleMember (and resolveModuleImportToFile for the file→file edge) name the target submodule by joining the import source with imp.localName. Under from pkg import module as alias that is the alias, so the lookup asks for pkg.alias — which names no module — and the edge is dropped. localName and exportedName coincide only when the import is unaliased, which is why the scratch cases in this thread passed and the real tree did not: minimal repros rarely alias, production code usually does.

    Minimal repro:

    pkg/__init__.py      (empty)
pkg/module.py        def func(): return 1
main.py              from pkg import module as mod_alias

                     def caller():
                         return mod_alias.func()

    caller gets no outgoing calls edge and codegraph callers func reports none. Removing as mod_alias makes both appear. Same for the file→file imports edge.

    Confirming it is resolution and not extraction: the reference rows are present and correct in unresolved_refs, with the receiver intact and status = failed — the extractor records pkg_alias.func, resolution just never matches it.

    Measured on a 991-file Python project that aliases module imports throughout: a target with 6 call sites across 5 enclosing functions went from 0 callers to 5, another went 2 → 4, and the index gained ~824 edges. Unaliased and class-based resolution were unchanged, and the full suite's failing-test set was byte-identical to unmodified main.

    PR: #1453

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions