Repository navigation
perf(template): cache the variables of a string template per template text - #1471
Open
ajtmccarty wants to merge 1 commit into
Open
ajtmccarty wants to merge 1 commit into
ajtmccarty wants to merge 1 commit into
Conversation
… text Reading the variables of a string template built a sandboxed environment and parsed the template on every call. get_template_variables() caches the sorted variable names per template text, and get_variables() uses it for string templates. A filter the template object lacks no longer raises when its variables are read; validate() and render() still reject it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ajtmccarty
added a commit
to opsmill/infrahub
that referenced
this pull request
Nov 2, 2026
…arate PR This reverts commit 22be5f8. The cache moves to the SDK (opsmill/infrahub-sdk-python#1471); a separate PR will update the submodule and use it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Deploying infrahub-sdk-python with
|
| Latest commit: |
a7c35e2
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://9f4d5392.infrahub-sdk-python.pages.dev |
| Branch Preview URL: | https://ajtm-11022026-template-varia.infrahub-sdk-python.pages.dev |
3 of 5 tasks
Codecov Report✅ All modified and coverable lines are covered by tests. @@ Coverage Diff @@
## infrahub-release-1.12 #1471 +/- ##
=========================================================
- Coverage 87.74% 87.74% -0.01%
=========================================================
Files 155 155
Lines 15517 15533 +16
Branches 2085 2087 +2
=========================================================
+ Hits 13615 13629 +14
- Misses 1345 1346 +1
- Partials 557 558 +1
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 1 file with indirect coverage changes 🚀 New features to boost your workflow:
|
ajtmccarty
added a commit
to opsmill/infrahub
that referenced
this pull request
Nov 2, 2026
…branch [IFC-3447] (#11252) * fix(number-pool): count a scoped number only in the divisions of its branch [IFC-3447] A scoped number pool read each scope element on any live branch, independently of the branch holding the number. A device moved to another site and renumbered on a branch counted both numbers in both sites, so allocation skipped free numbers and the division figures were too high. The division filter and listing now give each edge the list of live branches it is visible from, intersect the lists one scope element at a time, and count a number in a division only on a branch that holds the number and puts the Node in the division. A value the default branch holds is reported on the inheriting branches that put the Node in the division when the default branch does not. A reservation is read in the division the Node occupies on the request branch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(number-pool): compile the division queries on Neo4j 2025.10 [IFC-3447] Neo4j 2025.10 rejects an aggregating COLLECT subquery used as an operand of `+`, so every scoped-pool allocation failed with "Aggregation column contains implicit grouping expressions". Each division stage now projects only the attribute or the relationship subquery, chosen from the element's path, instead of adding the two. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(number-pool): narrow division-filtered queries to the division's candidates The free, used and allocated queries with a division ran the per-branch division check on every reservation of the pool. They now collect, once, the attributes of every node ever related to the division's first relationship value, and check other scope elements per attribute before the per-branch stages run. The branch lists read each edge's properties once. At 6k devices, the free-number query on a scoped pool goes from about 73 ms to 8-11 ms, and a device create on a scoped pool from about 95-103 ms to 31-42 ms. Results are identical on the test dataset. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(node): cache the variables of each display-label template Loading many nodes parsed the same display-label Jinja2 template once per node. The variables are now cached per template text. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(number-pool): build division candidates from indexed values and match on nodes A division made only of attributes had no candidate set, so every reservation of the pool went through the attribute check. When the division has no relationship value, the candidate set now starts from the first indexed attribute value (AttributeValueIndexed), bounded to the pool's node kind. A large-type attribute keeps the per-attribute check, because its values are not indexed. Both candidate sets now collect the matching nodes instead of all their attributes, which removes an expansion of every attribute of each node. EDGE_BRANCHES becomes a pair of WITH clauses that read the edge's properties once, and the element subqueries reuse it. At 8k devices, the free-number query on a role+status pool goes from 186k to 99k database hits, and on a site pool from 42k to 27k. Results are identical to the base commit on the test dataset. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(number-pool): read each scope attribute once per node A node reaches the same attribute through one HAS_ATTRIBUTE edge per branch that changed it, so the element value subquery read the attribute's values once for each of those edges. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * revert(node): move the display-label template-variable cache to a separate PR This reverts commit 22be5f8. The cache moves to the SDK (opsmill/infrahub-sdk-python#1471); a separate PR will update the submodule and use it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cache the variables of a string Jinja2 template per template text
Infrahub reads the variables of a display-label template each time it loads a node whose schema declares one. Performance testing of GraphQL queries found that this parsing took a large share of the response time (comment on opsmill/infrahub#11252). In review, the cache was moved from Infrahub into the SDK (review comment).
Before this change,
Jinja2Template.get_variables()built aSandboxedEnvironmentwith all filters and parsed the template on every call. Jinja2's own cache (Environment.cache) applies only to templates loaded by name through a loader, not toparse()orfrom_string().Changes
get_template_variables(template: str) -> list[str]ininfrahub_sdk.template. It returns the sorted variable names of a string template and caches them per template text (lru_cache, 1024 entries). It returns a new list on each call.Jinja2Template.get_variables()uses this function for string templates. File-based templates are still parsed on every call, because a file's contents can change.get_template()andvalidate()are unchanged and use the environment of eachJinja2Templateobject.Behavior change: unknown filters
To collect variables, Jinja2 compiles the template, and compiling fails with
No filter named '<name>'for any filter the environment lacks. The variables do not depend on which filters exist, so on a cache miss the function parses with the same environment type as a string template object and registers a placeholder only for filter names that the environment lacks. The placeholder is never used for rendering. If Jinja2 calls it while folding constants, it raises, and Jinja2 then leaves that expression unfolded.As a result, for a string template,
get_variables()no longer raisesJinjaTemplateSyntaxErrorwhen the template uses a filter that the template object does not provide.validate()still rejects that filter: every filter it allows is present in the object's environment.render()also still fails on it. An unknown test (is <name>) still raises.Measurements (local, two-variable template)
Jinja2Template(template=t).get_variables()get_template_variables(t)Tests
tests/unit/sdk/test_template.py(40 passed) covers:ruff,tyandmypypass.Follow-up
After this PR merges, a separate Infrahub PR against
release-1.12will update thepython_sdksubmodule and callget_template_variables()fromDisplayLabel,BaseNodeSchema.get_derived_field_paths()and the uniqueness violation message.🤖 Generated with Claude Code