perf(resolution): remove the indexing regressions since v1.6.0 (vscode stall, per-ref rescans) — graph byte-identical - #2072
Merged
Conversation
…tracking isLocallyBoundJsName (#1759, cd4e65b) scans a file for a parameter that binds a bare-called name. Its per-parameter pattern `(type)?(default)?\s*` let one `name: T = value` text split several ways, so when the name was absent the search backtracked through every combination. On vscode's markersModel.test.ts, whose helper takes nine typed, defaulted parameters, each `suite`/`test` scan took 30-40s — one pool worker held a whole resolution batch while the rest idled, a 70-400s stall per run. Each parameter now has exactly one parse: its first non-space character after the identifier picks the type, default or bare alternative. The alternatives match exactly the strings the old optional groups did, so the result is identical for every input (4M generated cases and 5.6k real vscode file/name pairs agree); the scan is 0.1-0.3ms on that file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#2034 (5eaa6fe) sent every Python named import whose path the import resolver cannot place — stdlib and third-party modules such as `from unittest import mock` — through findPythonModuleFile, which scans every `__init__.py` file node for a path suffix. It ran once per ref: on django that scan alone went from 0.6s to 1.4s of resolution CPU. The candidates for a module depend only on the module path, so they are now collected once per resolution context, in the same name-lookup order; the importing file is still excluded per call, so the first survivor is the node the scan returned. The memo drops with the other import-resolver memos. Resolution output on django is hash-identical; single-threaded resolution CPU there goes from 6.5-6.8s to 4.9s (v1.6.0: 4.5s). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
isSealedModule (#1719, bffd50e) masks a candidate file's comments and strings before asking whether it is a module that exports nothing. Import refs call it for every same-named candidate across the tree, so on vscode it masked nearly every TypeScript file once per resolver: 3.3s of masking in a single-threaded pass, paid again in each pool worker. The verdict is a pure conjunction, so it now asks the cheap questions first: the source contains `import`, no node in the file is exported, and there is no CommonJS export. Masking runs only for files that pass all three, which in practice are the few that can be sealed. The masker blanks text and never deletes it, so a file without `import` in its source has none in its masked code either. Resolution output is hash-identical on vscode and django. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ield matchTsThisFieldCall (#1496, cece072) resolves `this.<field>.<method>()` by finding the field's declaration in the enclosing class. It re-split the whole file, stripped comments line by line and compiled three patterns for every such call, although the answer depends only on the class and the field: 16.5s of a 148s single-threaded resolution pass on vscode. The first declaring line — `: typeof X`, `: X` or `= new X`, in that order per line — is now memoized per (class node, field), and the last few classes' stripped lines are kept, since calls arrive file by file. Owners are still tried in the same order and everything after the declaration lookup is unchanged. Resolution output is hash-identical on vscode and django. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The cross-tier pass (b1f40c5) scans every non-test JS/TS file whose text has a `.add(`, `.get(` or `.on(` — nearly all of vscode — with receiver- chain patterns such as `a.b.c.add('job')`. A chain could begin at any letter of an identifier, so each letter re-read the dotted chain behind it; on vscode the pass ran 35s on one pool worker and set the synthesis phase's wall. A match that starts mid-word implies one from the word's first letter, and every match of these patterns ends on a non-word character, so a scan never resumes mid-word: a lookbehind that rejects those starts skips only attempts bound to fail. Over all 14,135 vscode JS/TS files the five patterns return identical matches (positions and groups) before and after, in 8.8s instead of 32.8s; the pass itself drops from ~9.8s to ~5.6s CPU standalone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…cks first Interface members (#1638) and nested const-bound functions (#1669) made common names — `editor`, `model`, `dispose` — carry thousands of nodes. matchByExactName ran ten chained filters over that list for every reference, copying it once per rule before the source-reading checks ran, and the method-call strategies copied it again to put the call site's file first before looking at only the class-like or same-file entries. Each rule judges one candidate on its own, so the exact-name rules now run as a single pass with the kind and language checks ahead of the lexical, sealed-module and local-binding checks; the class strategies narrow to owner kinds before ordering by file (the same nodes in the same order), and the object-literal strategy, which keeps same-file holders only, skips the reordering altogether. Resolution output is hash-identical on vscode and django. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Eleven synthesis passes stream every node of a kind and keep only their language's (Go, Java/Kotlin, XML, native, ArkTS, Nix, Erlang). On a TypeScript monorepo the Go passes materialized all 113k vscode methods to find two Go ones — goMethodContains alone went from 0.5s to ~2.9s on the main thread after interface members (#1638) grew the method count and the canonical ORDER BY (#1988) landed. iterateNodesByKindIn adds the language predicate to the same query. Its ORDER BY (file_path, start_line, id) is a total order, so it yields exactly the nodes the callers kept, in the same sequence. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…side the passes Two serial main-thread steps landed with the incremental synthesis refresh (#1988, #2038). The bulk-edge window rebuilt idx_edges_synthesis_site before synthesis — its partial predicate runs json_valid over the metadata of every edge (~1.5M rows, 2.7s on vscode) — and after the merged insert synthesis re-read every file in the project to record its source gates (~1.9s). Neither result is read until the index is finished: only sync consults the index, and the gates depend only on the files. The window's end now leaves that one index for later, and the resolver builds it while the pool computes the passes, when the main thread was only waiting; the source-gate scan runs in the same window. Both still happen before resolution returns, whatever path synthesis takes, and a crash in between is healed on the next open as before. On vscode edge-index-recreate drops from 8.3s to 3.6s of serial time; the graph, synthesis_inputs and the final index set are identical. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…sing files Every resolved reference now re-reads its target node several times — the kind gate (#1796), the cross-family language gate (#1986), alias following (#1482) and edge creation — through the query layer's 1,000-entry node cache, while targets recur ~5x (1.46M vscode edges onto 293k nodes). The language gate alone spent ~6s of single-threaded resolution on vscode fetching whole nodes to read one field, and fileExists re-probed the same missing candidate paths (every extension of a `.js` specifier) from every importing file, now under a path.resolve containment check (#1631). The resolver keeps its own id → node cache and a memo of filesystem probes, both in the stable window of its other caches and dropped by clearCaches; the language gate returns early when the reference's language has no code family (no target can cross one) and otherwise memoizes each target's language. Resolution output is hash-identical on vscode and django. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
isSealedModule read and decoded a candidate file — its source and every node — to learn whether anything is exported, though nearly every module does. It now asks one indexed EXISTS probe first and reads the source only for files that export nothing (the conjunction is unchanged). isReceiverLessCall (#1759, #1857) compiled a pattern per call and end-anchored two more over the line's prefix, which V8 scans from the start. It now matches the name literally with a sticky `\s*[(<]`, and reads the prefix backwards: the last non-space character decides whether a receiver precedes the call, and the word run ending there decides the keyword exception — exactly what `[.\w$\])]\s*$` and `\b(?:return|...)\s*$` accepted. Resolution output is hash-identical on vscode and django. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…istry and tier scans The pooled fan-out handed every pass out at once, so each worker held a count-based share and a heavy pass finished only with its worker's whole share (vscode's registry pass ended ~16s in). Workers now pull one pass at a time, heaviest first; edges still merge by registry index, so the output is unchanged. vscode synthesis: 14.5s vs 19.6-24.4s. The registry pass retried its greedy name at every character of every identifier; both scans now start a name only at an identifier's first character or a `this.` (a mid-identifier resume after a `.method` dispatch falls back to the unguarded pattern), and dispatch lines come from a newline-offset lookup instead of splitting the prefix per match. Checked match-for-match against the old patterns on every vscode JS/TS file plus 300k fuzzed strings. The tier pass reads a file's nodes only when a site needs them. Standalone on vscode: registry 5.6s -> 4.5s, tier 5.1s -> 4.4s. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…field and local-binding checks
Three per-lookup costs the resolver pool paid once per worker:
- A file's export index decoded every node in the file and read its source,
for each file an import reached, in every worker. It now reads only the
exported rows (same `start_line, id` order); the default-export binding
and names a local `export { a as b }` clause introduces are read on first
use, from name-targeted rows. findExportedSymbol across vscode's workers:
12.5s -> 3.7s.
- The `this.<field>` declaration scan compiled three patterns per field of
every class and walked every class line. Each pattern needs the field as
a whole `[\w$#]` token, so a class's lines are indexed by token once and
three sticky patterns, compiled once, are tried at the field's token
occurrences. Identical on 721k vscode (file, field) pairs; 56.7s -> 1.1s
standalone.
- isLocallyBoundJsName searched the whole file with four patterns per name.
Each can only start at a `const`/`let`/`var` or `function`/`class`
keyword, an `=>`, or the `(` opening a list before the name, so the
patterns (compiled once per name, sticky) are tried there. Identical on
198k vscode (file, name) pairs; 68.1s -> 10.7s standalone.
Replaying every vscode (2.19M) and django (270k) reference single-threaded
gives the same result hash before and after.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…inherited this-member walk - getOutgoingEdges / getIncomingEdges with a kind filter prepared a fresh statement on every call, a third of the read's cost; they are issued per node by supertype walks and the member-lookup synthesis passes. Each SQL shape is now prepared once per connection (dropped on rebind). - The post-loop `this.<member>` pass walked the same supertypes for every reference, reading each one's contains edges and decoding members one by one until a name matched. Nothing is written until the pass ends, so each type's supertype edges and each supertype's callable members (by name, in edge order) are read once. vscode: 4.1s -> 1.0s. - Express's mount pass decoded every JS/TS file to look for `.use(`; it now checks the raw bytes first (an ASCII needle survives UTF-8 decoding unchanged) and decodes only files that have it. A full vscode init dumps the identical graph before and after. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ution never reads The batch loop inserted every batch's edges before fanning out the next one, so the pool sat idle through each insert. That ordering exists for one reason: later batches walk supertype chains over the `extends` / `implements` edges earlier batches resolved. Those are the only edges resolution reads, and the prerequisite phase drains every such ref first, so after it no batch makes one. A batch without them now fans the next batch out first and inserts while it resolves: the same writes in the same order, and nothing the workers read differs. The kinds live in one constant, shared with the supertype walk. vscode resolution, three interleaved rounds against the parent commit: 50.3-51.0s vs 56.8-59.1s wall (the pool's wait per batch halves; the overlapped inserts cost ~7s more CPU). Full vscode and django inits dump identical graphs before and after, and identical to main. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every edge insert checks its endpoints with a 500-id IN query, a chunk at a time, and prepared that statement afresh for each chunk. The batch loop now runs these inserts on the main thread while the pool resolves, so the full-size statement is prepared once; only a final partial chunk prepares ad hoc (as deleteReferencesByRowIds already does). Same SQL, same result. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Main indexes vscode far slower than v1.6.0. The worst cause was a pathological batch that could stall one pool worker for minutes, and several per-reference costs added since v1.6.0 made up the rest. This PR removes them and overlaps work the resolver pool used to sit idle through. The graph is byte-identical to main on both test repos.
Result
Fresh
codegraph init -y, kernel on, macOS arm64 (4 performance + 4 efficiency cores). Runs are interleaved with rotated order, and each one waits for a 1-minute load average under 3. The machine is shared with a Windows VM and other sessions, and a round where either arm was hit is dropped (noted below).vscode round 3 is dropped: the Windows VM woke up (99% CPU) mid-run and inflated both arms (v1.6.0 115s, branch 107s). Main was measured once per repo under the same load gate. Its vscode time varies run to run with how long the stalled batch takes.
Where the remaining time goes
It's mostly deliberate work added since v1.6.0, measured per phase on vscode:
method_signature/property_signaturemissing from the extractor) — every call through a.d.tsplatform API is invisible #1638) and declaration initializers (Walk declaration initializers scoped to the declared symbol — Kotlin, Java, TS/JS, Scala, Rust, Python (#693 for the other six languages) #1511).What was slow, and the fix
suiteandtestinmarkersModel.test.ts, whose helper takes ninename: T = valueparametersisLocallyBoundJsName's parameter regex backtracked exponentially(before an occurrence), and compiled once per namematchTsThisFieldCall: 16.5s of a 148s single-threaded resolvethis.x.m()call.jsspecifiers resolve to.ts, so 649k vscode refs now resolve through their importjson_valid) and the source-gate scan ran serially; language-specific scans read every nodethis.<member>pass: 1.9s → 3.9s.use(before decodingextends/implementsedges, and the prerequisite phase drains those refs first. So a batch that made none fans out the next batch before its insert: same writes, same order. Resolve wall −6s in the phase benchmark (three interleaved rounds). In a full init the overlapped inserts run slower, so the gain is smaller: about 1s, from separate runs rather than an A/BfileExistsmemo, kind-filtered edge statements and the endpoint-existence statement prepared onceEvidence that the output is unchanged
Full-index dumps (
scripts/dump-graph.mjs, freshinit, kernel on) are identical to main c3b1348:31865b7f…, 3,171,734 lines (495,223 nodes / 1,943,786 edges / 717,977 refs)129c7512…, 395,079 linesResolution replays take a
VACUUM INTOsnapshot right before resolution, re-resolve every ref single-threaded, and hash the output. The hash is identical before and after each resolver change: vscode 2.19M refs6c58d536…, django 270kcfeb6994….Every rewritten matcher has a differential test against the old code:
Tests: full suite, 5,194 passed / 234 skipped / 0 failed.
Also noted
this._logService.warn/.trace/.info/.errorcalls now fail through interface-typed fields.git status(fix(status): preserve pending changes across commits and restores #1878) re-hashing acp-copied tree with a stale stat cache: 3.7s there vs 0.16s on a real clone. No code change.🤖 Generated with Claude Code