Repository navigation
perf!: Replace the OrderedSet children container with a flat sorted-array ComponentList - #4019
Conversation
luanpotter
left a comment
There was a problem hiding this comment.
This is great and super clean @spydon thanks for breaking it up.
I left some comments but have 3 overarching comments:
-
as we agreed before, dart's for each (I mean the language construct- not the lambda) compiles just the same as the indexed for
-
I am considering if we can extract a mixin, base class, whatever from Component to hold the "bears children" logic - it would include all the 3 related fields (the children, the parent list ref, and the index). these are the only things
ComponentListcares about for Component specifically (other than the default priority comparator that can move tocreateComponentList. that wayComponentListbecomes truly component agnostic, and the component class can remove some of this related index tracking and children logic. not sure it would be as clean as I am envisioning, but worth investigating. -
I am curious about the efficacy of our benchmarks in real life:
All components are plain [Component]s with no-op [Component.update] and [Component.render], so any time measured here is framework overhead rather than game logic.
while yes that isolate engine overhead, I wonder if the magnitude is even relevant on real cases (I suspect yes, but not 100% sure). would be awesome to run the same bench with some bog-standard possible implementations of updates, like a light and a heavy one, to see how the optimizations fare into a real game.
but overall, all things being said - I am very happy with this direction :)
1d403f2 to
848b26c
Compare
Yes, for-in, not for each. I'll fix that up. :)
It sounds like overengineering to me... It's not that much overhead in the Component, and it really is the core of the Component.
Setting up more "real life" benchmark too and adding the results to the description. |
…ory with per-component comparator support
…ual-priority ordering
…teComponentList Claude-Session: https://claude.ai/code/session_01TAzCZ5NMKo7VhLML7Wgxdn
Use for-in loops over the backing list, make reversed a getter, drop the Component.strictQueryMode static in favour of the constructor default, share one binary search helper, document the container fields and the strict-mode error, and drop the historical comparisons from the docs. Claude-Session: https://claude.ai/code/session_01XJydxtxerc3QG6hJTf3dC1
They pin ordering behavior with plain expectations, not golden images. Claude-Session: https://claude.ai/code/session_01XJydxtxerc3QG6hJTf3dC1
…nternalChildren ComponentList is a part of component.dart, so add, remove and clear can be private instead of @internal. The _internalChildren getter was identical to the children getter. Claude-Session: https://claude.ai/code/session_01XJydxtxerc3QG6hJTf3dC1
Delegate priority ordering to compareTo, use contains in the private mutators, avoid a null assertion in componentsAtLocation, and move the container migration guide sections into this PR. Claude-Session: https://claude.ai/code/session_01XJydxtxerc3QG6hJTf3dC1
The lifecycle queue mutators, the lifecycle event status, and the per-event handlers on Component are now private instead of internal. ComponentList.rebalance is private too; Component.rebalanceChildren stays the public entry point for custom comparators. The public surface of package:flame/components.dart is unchanged. Claude-Session: https://claude.ai/code/session_01XJydxtxerc3QG6hJTf3dC1
a91adb0 to
01f0ec2
Compare
| import 'package:flame/components.dart'; | ||
| import 'package:flame/src/components/core/recycled_queue.dart'; | ||
| import 'package:meta/meta.dart'; | ||
| part of 'component.dart'; |
) Second half of the FCS core rewrite for #3957: replaces the recursive, virtual `updateTree` with a root-owned flattened traversal list. **Stacked on #4019.** That PR replaces the `OrderedSet` children container with the flat sorted-array `ComponentList` (O(1) remove and contains, tombstone compaction, one stable sort per parent per tick, `createComponentList()` with an optional comparator). This PR is based on that branch and only contains the traversal work on top of it; the diff here shrinks to the traversal commits, and the container change is reviewed over there. Once #4019 lands, this PR retargets to `main` on its own. ## The update pass runs over a flattened traversal list `Component.updateTree` is now `@nonVirtual`. Components that manage their own subtree traversal mix in `CustomTraversal` and override its `updateSubtree`; the method only exists on the mixin, so it cannot be overridden without also becoming a barrier. Traversal mixins such as `HasTimeScale` are declared `on CustomTraversal` and compose through `super.updateSubtree`, which means their users write `with CustomTraversal, HasTimeScale` (except on `FlameGame`, which already mixes it in). The root updates everything through a flattened pre-order list that is rebuilt lazily, only on ticks where the tree structure changed (and then fused into that tick's update pass, so the rebuild costs no extra traversal). `CustomTraversal` components act as barriers that drive their own subtrees. The render pass intentionally stays recursive and virtual, since `renderTree` has many legitimate overriders (decorators, visibility, snapshot, cameras). `Route.stopTime()` and `resumeTime()` keep their public behavior. The gating now lives in `HasTimeScale` itself: a `timeScale` of `0` stops the update pass for the whole subtree instead of updating it with a `dt` of `0`, so `Route` no longer needs an override. `FlameGame` processes lifecycle events before dispatching into the traversal, so components added to a paused game are still mounted. ## Relation to the allocation-hygiene PRs The tangential hot-path improvements were extracted from this PR and reviewed independently, per [the review feedback](#3957 (comment)). Merged: #3978 (lifecycle early-out), #3979 (camera render closures; the decorator tear-off caching was deferred during its review), #3980 (lazy render contexts), #3981 (tree teardown and propagation), #3983 (sweep broadphase). #3982 (pointer-handler counters) was closed in favor of an upcoming events rework. This branch includes all of that, and on top of #4019 it contains only the traversal work. The golden tests added in #4019 (lifecycle-event ordering, hit-test order, equal-priority ordering) run unchanged over the flattened traversal. ## Benchmarks (full stack: #4019 plus this PR, against main) Measured against current main, which already includes the merged hygiene PRs, so these tables isolate the effect of the two stacked PRs together. The per-PR attribution follows in the next section. ms per run, lower is better. ### JIT (`flutter test`, host machine, repeated best-of runs) | Benchmark | main | this PR | speedup | |---|---|---|---| | Update wide tree (10k x 1) | 8.65 | 3.66 | 2.4x | | Update nested tree (1k x 10) | 22.45 | 4.00 | 5.6x | | Update deep tree (100 levels) | 18.81 | 3.41 | 5.5x | | Update barrier tree (10% time-scaled) | 21.04 | 4.51 | 4.7x | | Render wide tree (10k x 1) | 12.63 | 7.17 | 1.8x | | Render nested tree (1k x 10) | 22.66 | 10.80 | 2.1x | | Render deep tree (100 levels) | 24.96 | 9.47 | 2.6x | | Lifecycle churn (100/tick, 1k pop) | 4.02 | 2.74 | 1.5x | | Lifecycle churn (100/tick, 10k pop) | 11.42 | 9.24 | 1.2x | | Mass add/remove (1k per cycle) | 2.47 | 1.54 | 1.6x | | Priority change (1 child per parent) | 75.10 | 7.08 | 10.6x | | Priority change (y-sort, all children) | 23.25 | 6.37 | 3.6x | | Type-query churn (2 registered queries) | 6.73 | 5.85 | 1.2x | | Game-like update pass | 202.71 | 107.48 | 1.9x | | Hit test + delivery (cached) | 84.24 | 77.25 | 1.1x | | Flat collision detection | 9.57 | 3.35 | 2.9x | | Nested collision detection | 15.59 | 3.36 | 4.6x | ### AOT (`flutter run --release`, Android arm64 emulator on an Apple Silicon host, single run) | Benchmark | main | this PR | speedup | |---|---|---|---| | Update wide tree (10k x 1) | 6.25 | 1.45 | 4.3x | | Update nested tree (1k x 10) | 9.53 | 1.13 | 8.4x | | Update deep tree (100 levels) | 11.28 | 1.67 | 6.8x | | Update barrier tree (10% time-scaled) | 12.35 | 1.39 | 8.9x | | Render wide tree (10k x 1) | 6.42 | 3.96 | 1.6x | | Render nested tree (1k x 10) | 12.44 | 4.92 | 2.5x | | Render deep tree (100 levels) | 14.74 | 6.61 | 2.2x | | Lifecycle churn (100/tick, 1k pop) | 1.54 | 1.07 | 1.4x | | Lifecycle churn (100/tick, 10k pop) | 4.80 | 4.52 | 1.1x | | Mass add/remove (1k per cycle) | 0.89 | 0.44 | 2.0x | | Priority change (1 child per parent) | 24.67 | 4.15 | 5.9x | | Priority change (y-sort, all children) | 11.28 | 3.31 | 3.4x | | Type-query churn (2 registered queries) | 5.36 | 4.73 | 1.1x | | Game-like update pass | 79.68 | 21.42 | 3.7x | | Hit test + delivery (cached) | 43.30 | 37.42 | 1.2x | | Flat collision detection | 4.26 | 1.01 | 4.2x | | Nested collision detection | 7.92 | 0.97 | 8.2x | The AOT numbers come from the `crud_test` AVD (API 34, arm64-v8a, 4 cores), which executes the AOT snapshot natively on the host CPU, so they are representative of release-mode Dart rather than of any particular phone. The earlier guess that AOT would show smaller multiples turned out to be wrong for the update pass: the four update-pass rows and both collision rows gain more under AOT than under JIT (nested tree 8.4x, barrier tree 8.9x, nested collision 8.2x), the render and churn rows land about where JIT put them, and priority change gains less (5.9x against 10.6x). The ordering of the rows is therefore not the same as in the JIT table; the update pass is where AOT rewards the flat list the most. ## Attributing the speedup: how much comes from the container, how much from traversal flattening The stack bundles two structural changes: the `OrderedSet` to `ComponentList` container rewrite (#4019), and the flattened-traversal update pass built on `CustomTraversal` (this PR). To see how much each contributes on its own, the benchmark suite was run at three points in the branch history, on the same machine. - `main` (commit `7fd33dee9`): the baseline, before either PR. - `1a25aea1f`: the container rewrite alone, which is exactly the head of #4019. It has the flat sorted array `ComponentList`, O(1) remove and contains, tombstone compaction, and a single stable sort per parent per tick, with the old recursive, virtual `updateTree` still in place. - `HEAD`: the full stack, container rewrite plus traversal flattening. Two of the later commits on this branch were small allocation cleanups to `ComponentList` that built on the traversal-flattening commit, so they could not be isolated from the traversal work and are counted on that side of the split. The query-cache list mirror they introduced has since been dropped again: iterating the map values directly measures the same on the type-query churn benchmark. ### JIT (`flutter test`, host machine, single run) | Benchmark | main | container only | this PR | container gain | traversal gain | |---|---|---|---|---|---| | Update wide tree (10k x 1) | 3.24 | 1.19 | 1.28 | 2.7x | flat | | Update nested tree (1k x 10) | 8.08 | 2.57 | 1.35 | 3.1x | 1.9x | | Update deep tree (100 levels) | 8.46 | 3.23 | 1.21 | 2.6x | 2.7x | | Update barrier tree (10% time-scaled) | 9.15 | 3.85 | 1.69 | 2.4x | 2.3x | | Render wide tree (10k x 1) | 5.14 | 4.05 | 3.98 | 1.3x | flat | | Render nested tree (1k x 10) | 9.95 | 4.92 | 4.64 | 2.0x | flat | | Render deep tree (100 levels) | 12.34 | 6.56 | 5.45 | 1.9x | 1.2x | | Lifecycle churn (100/tick, 1k pop) | 1.80 | 1.15 | 1.34 | 1.6x | flat | | Lifecycle churn (100/tick, 10k pop) | 4.86 | 2.82 | 4.64 | 1.7x | flat | | Mass add/remove (1k per cycle) | 1.15 | 0.64 | 0.66 | 1.8x | flat | | Priority change (1 child per parent) | 26.01 | 3.44 | 3.73 | 7.6x | flat | | Priority change (y-sort, all children) | 11.72 | 3.33 | 3.35 | 3.5x | flat | | Type-query churn (2 registered queries) | 3.41 | 2.83 | 2.94 | 1.2x | flat | | Game-like update pass | 98.34 | 70.09 | 42.62 | 1.4x | 1.6x | | Hit test + delivery (cached) | 43.63 | 41.93 | 53.24 | flat | noisy | | Flat collision detection | 4.95 | 2.04 | 1.70 | 2.4x | 1.2x | | Nested collision detection | 7.91 | 2.84 | 1.83 | 2.8x | 1.6x | ### AOT (`flutter run --release`, Android arm64 emulator, single run) | Benchmark | main | container only | this PR | container gain | traversal gain | |---|---|---|---|---|---| | Update wide tree (10k x 1) | 6.25 | 4.31 | 1.45 | 1.5x | 3.0x | | Update nested tree (1k x 10) | 9.53 | 3.91 | 1.13 | 2.4x | 3.5x | | Update deep tree (100 levels) | 11.28 | 4.27 | 1.67 | 2.6x | 2.6x | | Update barrier tree (10% time-scaled) | 12.35 | 6.81 | 1.39 | 1.8x | 4.9x | | Render wide tree (10k x 1) | 6.42 | 3.97 | 3.96 | 1.6x | flat | | Render nested tree (1k x 10) | 12.44 | 4.70 | 4.92 | 2.6x | flat | | Render deep tree (100 levels) | 14.74 | 6.53 | 6.61 | 2.3x | flat | | Lifecycle churn (100/tick, 1k pop) | 1.54 | 0.90 | 1.07 | 1.7x | 0.8x | | Lifecycle churn (100/tick, 10k pop) | 4.80 | 3.08 | 4.52 | 1.6x | 0.7x | | Mass add/remove (1k per cycle) | 0.89 | 0.42 | 0.44 | 2.1x | flat | | Priority change (1 child per parent) | 24.67 | 3.42 | 4.15 | 7.2x | 0.8x | | Priority change (y-sort, all children) | 11.28 | 3.30 | 3.31 | 3.4x | flat | | Type-query churn (2 registered queries) | 5.36 | 4.75 | 4.73 | 1.1x | flat | | Game-like update pass | 79.68 | 50.16 | 21.42 | 1.6x | 2.3x | | Hit test + delivery (cached) | 43.30 | 38.48 | 37.42 | 1.1x | flat | | Flat collision detection | 4.26 | 1.47 | 1.01 | 2.9x | 1.5x | | Nested collision detection | 7.92 | 1.95 | 0.97 | 4.1x | 2.0x | The container rewrite alone accounts for most of the gain across the board: nearly all of the priority change, lifecycle churn, mass add and remove, and render benchmarks, and roughly half of the update pass benchmarks. The traversal flattening adds most of its own value on trees with real nesting or depth (nested tree, deep tree, barrier tree, both collision detection benchmarks, the game-like update pass), where recursion overhead is what is left to cut once the container is fast. On the flat wide tree and on the simpler priority change and churn benchmarks, the traversal flattening contributes close to nothing on top of the container change, which fits, those do not stress recursive traversal in the first place. Under AOT the split is the same in shape but the traversal side is worth more: the flattened traversal is a 2.6x to 4.9x gain on top of the container on every update-pass row, including the flat wide tree where it was flat under JIT, and 2.3x on the game-like update pass. The render rows are flat on the traversal side in both modes, as expected since the render pass stays recursive. One trade-off shows up in both modes and is worth calling out: lifecycle churn at 10k population is slower with the flattened traversal than with the container alone (JIT 2.82 to 4.64 ms, AOT 3.08 to 4.52 ms), and single-child priority changes lose a little (AOT 3.42 to 4.15 ms). Both benchmarks change the tree structure every tick, so the flat list is rebuilt every tick and the rebuild cost is paid without ever being amortized. Against `main` both rows are still ahead or level (churn at 10k: 4.80 to 4.52 ms AOT), and a real game changes structure on a fraction of its ticks, but it is the one workload where this PR gives some of the container's gain back. These attribution numbers come from a single run per checkpoint, so treat the exact multiples loosely, particularly the rows marked flat or noisy. Hit test and delivery moved within noise across all three checkpoints in both runs and should not be read as a regression. --------- Co-authored-by: Luan Nico <luanpotter27@gmail.com>
Description
First half of the FCS core rewrite for #3957: replaces the
OrderedSetchildren container with a Flame-ownedComponentList. Split out of #3960 so that the container change can be reviewed on its own; #3960 is stacked on top of this branch and adds the flattened update traversal.Children live in a Flame-owned
ComponentListThe
ordered_setdependency is gone. Children are stored in a single flat array sorted by(priority, insertion order), and each component intrusively stores its container and slot index:removeandcontainsare O(1), with no hashing or tree walks;addis an O(1) append in the common case.nulltombstones that are compacted once per parent per tick, so removing k children among n costs O(k + n) instead of O(k*n).register<T>()/query<T>()cache surface is kept, and results are now always in priority order.Component.childrenFactoryis replaced by an overridablecreateComponentList(), which accepts an optionalComparator<Component>for custom orderings such as y-sort.The update, render, and hit-test passes iterate the backing array directly instead of going through an iterator, but the traversal itself is unchanged:
updateTreeis still recursive and virtual here.component_tree_ordering_test.dartpins lifecycle-event ordering, hit-test order, and equal-priority ordering across the rewrite.Three backing designs were implemented and benchmarked before settling on this one; see the comparison in the issue.
Benchmarks
Measured against
main(commit7fd33dee9, which already includes the merged allocation-hygiene PRs), so both tables isolate this PR's own effect. The benchmark suite is inpackages/flame/benchmark; ms per run, lower is better.JIT (
flutter test, host machine)AOT (
flutter run --release, Android arm64 emulator on an Apple Silicon host)Both tables are single runs per checkpoint (the JIT one is the
container onlycolumn of the attribution run in #3960), so treat the exact multiples loosely. The AOT numbers come from thecrud_testAVD (API 34, arm64-v8a, 4 cores), which executes the AOT snapshot natively on the host CPU, so they are representative of release-mode Dart rather than of any particular phone.Update pass with real game logic (JIT and AOT)
@luanpotter asked whether the traversal gains above still matter once components do real work, so
update_workload_benchmark.dartreuses the wide and nested trees from the traversal suite but gives every component a bog-standard, allocation-freeupdatebody:position.addScaled(velocity, dt)plus bouncing off the world bounds.atan2facing, and a cooldown timer.The no-op rows are the traversal benchmarks re-run in the same session for reference. Light rows do 500k component visits per run and heavy rows 100k (the no-op rows do 1M), so compare each row against its no-op counterpart rather than across rows. Median of 3 alternating runs, ms per run, same machines as the tables above (
mainatb4578fc6a).Per component visit in AOT, the framework overhead on
mainis about 4.5ns in the wide tree and 9ns in the nested tree, against roughly 25-45ns for the light update body (most of which is theNotifyingVector2change notification that marks the transform dirty) and 110-130ns for the heavy one. So onmainthe traversal accounts for around 18% (wide) to 21% (nested) of a light-logic update pass and 4-7% of a heavy-logic one, and this PR removes most of that: the whole update pass gets 10-20% cheaper with light logic and 4-10% cheaper with heavy logic. The nested shape, which is what real games look like (entities holding a handful of child components), is where the gain is largest, and the heavy-logic rows are the floor: the container change cannot speed up the game's own code, so a game that is bound by its ownupdatebodies sees single-digit percent gains from this PR on its own. The traversal flattening in #3960 stacks on top of these numbers.Checklist
docsand added dartdoc comments with///.examplesordocs.Breaking Change?
Migration instructions
childrenis now aComponentListinstead of anOrderedSet. The iterable surface,query<T>(), andregister<T>()are unchanged, so most code compiles as is. Theordered_setdependency is gone.children.reversed()is now a getter:children.reversed.Component.strictQueryModeis removed. Strict mode is off by default; to enable it for a component, overridecreateComponentList()to returnComponentList(strictMode: true).Component.childrenFactoryis removed. OverridecreateComponentList()on the component instead; it accepts an optionalComparator<Component>for custom orderings (for example y-sort).childrenwhile iterating it now tolerates removals and tail appends; only position-shifting operations (mid-list insertion, reorder, compaction) throwConcurrentModificationError.Related Issues
Part of #3957 (closed by #3960, which is stacked on this PR).