Repository navigation
fix(cpp): a call through a template parameter links to each class written as its argument - #3196
Open
colbymchenry wants to merge 1 commit into
Open
colbymchenry wants to merge 1 commit into
colbymchenry wants to merge 1 commit into
Conversation
…tten as its argument `Txn::PutEntity(…)` in `template <typename Txn> class SecondaryIndexMixin` names a member of whatever class the template is given, so no lookup at the call can answer it. 1.6.2 reached three such calls on rocksdb by name coincidence, and the boundary fix that ended that rule left them failed. What decides the target is written elsewhere, in `new SecondaryIndexMixin<WriteCommittedTxn>(…)`, and with several instantiations each really runs its own, so every class gets an edge. A new pass, cppTemplateEdges, reads the failed references that go through a template parameter: `T::x`, `T::T/2`, what `T x;` constructs, an alias of a member of the parameter, and a call on a value of its type. It finds what the template is written with: base clauses, variables, members, `new`, aliases, explicit instantiations, `f<Foo>()`, deduced `f(x)` where the argument's type is known exactly, gtest's type lists, and arguments that are another template's parameter, four templates at most. An argument is a project class by the lookup a declared type gets, a class local to the function first, in a file that sees it; anything else gives no edge and takes nothing from the rest. A specialization takes its instantiations from the template. A parameter that reaches more than 40 classes gets no edges: on seven repos that cut one reference, folly's OptionalStorage. The hop names the argument, the parameter and where it's written, in explore's Flow and the node trail. A walk over every file now has ResolutionContext.scanFile, which stays out of the content cache and skips UTF-8 decoding. Validation against main: all three rocksdb entries reached, 106 tests, 87 of them red with the pass off. Added edges: rocksdb 347, abseil 1,013, protobuf 969, folly 1,978, json 479, fmt 254, googletest 114, leveldb 3, gin and flask 0. Nodes, unresolved refs and every other edge are identical on all ten repos, with column and metadata. Pool, sequential and repeat runs give the same edges in the same order, and sync matches a fresh index on rocksdb, abseil and fmt. rocksdb index, 10 interleaved rounds: CPU 107.1 to 108.1 s mean, +1.0% (median +1.2%), peak RSS -0.5%, wall -0.1%. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A call through a C++ template parameter now links to that member of each class the project writes as the template's argument, one edge per class.
Txn::PutEntity(…)intemplate <typename Txn> class SecondaryIndexMixinnames a member of whatever class the template is given, so no lookup at the call can answer it. 1.6.2 reached three such calls on rocksdb by name coincidence. #2723's boundary fix took that rule away, correcting 4,219 edges, and it stays gone. What decides the target is written elsewhere:new SecondaryIndexMixin<WriteCommittedTxn>(…). With several instantiations each really runs its own, so a pick among them is a guess and all of them is the truth.What it does
cppTemplateEdgesis a new pass inSYNTH_PASSES, gated on C++, in a file of its own (src/resolution/cpp-template-instantiations.ts). It reads the failed references that go through a template parameter, finds what each template is written with, and gives each reference one edge per class. Edges areprovenance: 'heuristic',metadata.synthesizedBy: 'cpp-template-argument',registeredAtthe instantiation, and the hop reads, incodegraph_explore's Flow and thecodegraph_nodetrail:The three entries
lost/36Txn::PutEntitySecondaryIndexMixin::AddPrimaryEntry → WriteCommittedTxn::PutEntity@pessimistic_transaction.h:291, vianew SecondaryIndexMixin<WriteCommittedTxn>atpessimistic_transaction_db.cc:209lost/35Traits::ClearOutputGetFromBatchImpl→ both localTraits:GetFromBatch::Traits::ClearOutput(:920) andGetEntityFromBatch::Traits::ClearOutput(:991)lost/25TypeParam::KeyGenTYPED_TESTbodies → the sixKeyGenmember aliases of the 31 listed types that declare one (:213, :309, :319, :324, :332, :336),TypesAndSettings_Hash32_SmallKeyGen::KeyGenat :336 among themAn edge's line is the first call in the function, as with every synthesized edge: the merge keeps one edge per source and target.
How much it reaches
Not almost nothing: 197 references on rocksdb, 1,745 on folly, and real flows among them,
ShardedCache → LRUCacheShardon rocksdb andparse_chrono_format→ each handler on fmt.Why it abstained:
int,std::string, a pointer and the like, or a class no lookup finds. 1,325 of rocksdb's are one shape, its vendored gtest'sValueArrayNconversions.std::vector<T>&) or the argument's type isn't known. Protobuf's 2,094 are mostly test helpers such asTestUtil::ExpectAllFieldsSet<TestAllTypes>, given generated classes the checkout doesn't hold.ValueTable, whose base isInstantiateRawHashSet<…>::type, a member alias of a template.The cap
40 classes per parameter, chosen from the census with no cap at all on seven repos. The sizes run 52, 38, 38, 37, 37, 31, 31, 25, 24, 22, 22, 21, 21, and the next is 19. 64 would cut nothing, 32 would cut 10 references of real families (protobuf's
GetNewImpl, folly'sTry), 24 would cut rocksdb's own 31-type ribbon list. 40 sits in the gap and cuts exactly one reference on all seven:folly::detail::OptionalStorage<Value>'s placement new atfolly/Optional.h:149, 52 classes, a container. It gets no edges, not some. The cap is per parameter: a parameter under it is linked in full whatever a sibling parameter does.Forms of an instantiation
From the census, 20,307 sites across the seven repos, each a parameter the pass followed:
Mixin<Foo>::Build()usingortypedefnewtemplate class Mixin<Foo>;An instantiation inside another template is followed to that template's arguments, four templates at most, each parameter once, so a cycle ends.
What I changed from the design, and why
T::x. Failed calls on a variable of the parameter's type outnumber those on every repo (rocksdb 172 against 80, folly 2,272 against 246), and they are the flows:shard.SetCapacity(…)on aCacheShard& shard,handler.on_year(…)on aHandler& handler. The pass types the receiver the way a call is typed, and a->on a value is the argument's ownoperator->, so that is left alone.T,const T&,T&&orT*and the argument has a written type. C++ deduces exactly that, so the result is exact. Anything else is counted and left.CppNameSite.seesnarrowscppClassWritten: nlohmann/json'stests/abiholds a vendored copy of the library, and two abseil headers each declare aConstructorTestfor separate translation units. The first version sent map tests to set classes (736 edges against the true 475) until this went in.<anonymous>: a full or partial specialization takes its instantiations away from the template, and its own members read their parameters from what it matched (FooforTofName<T*>givenName<Foo*>). A condition (enable_if_t<…>) can't be read from text, so it counts as taken, which costs an edge rather than adding a false one.cppEdgesorcFnPtrEdgesbridges keeps its edge.Proof
Tests.
__tests__/cpp-template-instantiations.test.ts: 106 pass, 87 fail with the pass off (the other 19 say what must not link). Each form, an outside argument, the cap on both sides, adding, changing, removing and deleting an instantiation in a third file against a fresh index, pool against sequential. Mutation: 62 guards broken one at a time, 60 killed, and the two left are a prefilter and a fast path that change no result. That run sharpened a dozen tests and took out three guards that proved redundant. A fourth it called redundant wasn't: json's A/B showed 39 edges into its vendored copy without it, since a lookup through an alias falls back on what it can't see. It went back with a fixture that fails without it.A/B against main, compared in full: every node, every edge with its column, metadata and provenance, every unresolved ref.
Every added edge is
cpp-template-argument. leveldb's control isn't byte-identical on purpose: its threeinstantiatesedges (NoDestructor,SingletonEnv) are true.Classes added, on the whole diff:
callsmethod → methodcallsfunction → methodcallsclass → methodinstantiatesmethod → classinstantiatesfunction → classinstantiatesfunction → type aliasreferencesmethod → methodRead against source: 40 on folly, 36 on protobuf, 30 on rocksdb, 30 on fmt, 26 on json, 70 on abseil and leveldb's 3, drawn across every source, kind and parameter: all real. Each repo's edge set is the one those samples came from, except seven folly pairs
cppEdgesalready bridges, which keep its edge. Mechanical checks over all edges (the call line, the site, the member's owner) flagged only aliases andT::_Internal::New-style nested types, which read right. One note: protobuf'sHasBits::emptylands on the member of aHasBits<1>specialization, the template's name being all the argument says.Determinism. Pool, sequential and a repeat: identical edges in identical id order on rocksdb (347 of 342,405), fmt, abseil and json.
Sync. Remove an instantiation, change a typed list, add a third file, then compare every edge with a fresh index of the same tree: identical on rocksdb (all 342,294), abseil and fmt. A file holding only
template class Mixin<Foo>;now triggers the pass (hasSynthesisPattern).Cost on rocksdb, cold index, 10 interleaved rounds, measured as REG did:
CPU sits on the 1% line: paired, +1.1 s ± 0.6 of 107 s. The pass itself takes 0.57 s on a worker. Main got about a quarter faster under this work (#3182, #3183), which is what moved it from +0.7% to here, so the same pass now costs a bigger share. Peak memory comes at the end of
cFnPtrEdgesand moves between two modes about 90 MB apart from run to run. A rotated bench of a no-op pass and the full pass couldn't tell them apart (against main, no-op +17 MB ± 20, full -20 MB ± 27).A file that doesn't write a template's name is walked once for its names and never parsed for it. That walk cost more than it looked: it went through the content cache and pushed out what other passes wanted, and a name kept from a file's text keeps the whole text alive (V8 sliced strings).
ResolutionContext.scanFile(no cache, no stat, no UTF-8 decode) anddetachedfixed both, and took the pass alone from 2.8 s to 1.6 s cold and 0.75 s to 0.36 s warm. The walk of the failed refs reads a whole table, so it empties the connection's page cache after, which gives back about 33 MB on rocksdb. Caches are bounded at 4,096 files and 1,024 templates for projects larger than these.Explore connects on rocksdb and fmt, output above. No extraction change, so no kernel parity run.
Suite on the exact commit, 4 workers,
CODEGRAPH_KERNEL_EXPECT=1: 884 files, 10,976 passed, 0 failed, 87 skipped, 2 expected fail.🤖 Generated with Claude Code