Repository navigation
perf: reduce allocations when pushing scopes via ILogger.BeginScope - #5828
Conversation
…xts and attachments Scope.Clone runs on every ILogger.BeginScope call made while Sentry's logging integration is registered. Every Scope eagerly allocated three ConcurrentDictionary instances (tags, extras and the one inside SentryContexts), which made up around 80% of the 4.4 KB each scope cost on a 20 core machine (about 60% of 2.5 KB with 2 cores). Create the tags, extras, contexts, attachments and SDK version lazily, and skip copying them in Clone and Apply when the source has none. The public properties still return the same live instance once created. Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Code <noreply@anthropic.com>
…ocations Scope.Clone enumerated the breadcrumbs of the source scope (which takes a snapshot array) and enqueued them one by one into the clone, repeatedly growing the clone's backing list. With the default of 100 breadcrumbs that was around 3 KB per clone. Copy the items of the source queue directly into the clone with a single sized allocation instead. Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Code <noreply@anthropic.com>
…e used SentryLogger.Log invoked the formatter before checking whether the entry came from the Sentry SDK itself, was an EF exception message or was excluded by a log entry filter, so the message was formatted (and allocated) for every entry that was then discarded. The filter check also allocated a closure for every entry through LINQ's Any(). Move the formatting after those checks and replace the LINQ call with a loop. Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Code <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #5828 +/- ##
==========================================
+ Coverage 76.27% 76.32% +0.04%
==========================================
Files 529 528 -1
Lines 19523 19528 +5
Branches 3810 3826 +16
==========================================
+ Hits 14892 14905 +13
+ Misses 3752 3744 -8
Partials 879 879 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
ric-oliv
left a comment
There was a problem hiding this comment.
Hi @martincostello, thank you for the contribution!
This is a nice win for apps that open a lot of logging scopes, and the benchmarks in the description made it easy to follow.
I left a few comments. The one I'd judge more important is the extra asserts in the clone test. The LazyInitializer suggestion and the test cleanup are optional, so take them or leave them as you / @jamescrosswell prefer.
Use LazyInitializer.EnsureInitialized for the lazily created collections of Scope instead of hand-written compare-and-swap loops. Assert in the Clone test that the contexts and attachments of a clone are independent of the scope they were cloned from, and remove new tests that only duplicate the coverage of existing tests. Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Code <noreply@anthropic.com>
|
@ric-oliv Thanks - the latest commit should resolve your feedback. |
Nice - thanks @martincostello 🙏🏻 The repo already has a Not a blocker though - we can do that as a follow up so as not to hold this PR up. |
Description
Every call to
ILogger.BeginScopemade while Sentry's logging integration is registered callsHub.PushScope, which clones the currentScope. Applications that create logging scopes for each request or message therefore clone a scope many times a second. In a Pyroscope profile of an ASP.NET Core application under a load test,Scope.Clone(and theScopeconstructor) was the largest single source of allocations, at about 21% of all the bytes allocated by the process, so this PR reduces what a scope costs to create and clone. Each change is in its own commit.Create the collections of a
ScopelazilyEvery
Scopeeagerly allocated threeConcurrentDictionaryinstances (tags, extras and the one insideSentryContexts), which made up around 80% of the 4.4 KB each scope cost on a 20 core machine, as well as the attachments and the SDK version. The tags, extras, contexts, attachments and SDK version are now created on first use, andCloneandApplyskip copying them when the source has none. The public properties still return the same live instance once it has been created, including when first read concurrently. Reading one of the properties creates the (empty) collection, so code that reads them on every scope pays for it on the first read.Copy breadcrumbs without intermediate allocations in
Scope.CloneCloneenumerated the breadcrumbs of the source scope (which takes a snapshot array) and enqueued them into the clone one by one, repeatedly growing the clone's backing list. With the default of 100 breadcrumbs that was around 3 KB for each clone. A newConcurrentQueueLite.EnqueueRangecopies them with a single, exactly sized allocation instead. Ordering and the maximum number of breadcrumbs are unchanged.I considered making
CloneO(1) with copy-on-write breadcrumbs, but that means replacing the list-backed queue with a persistent or chunked structure, which changes breadcrumb ordering, trimming and thread-safety semantics, so I left it out.Only format a log message once it is known to be needed in
SentryLoggerSentryLogger.Loginvoked the formatter before checking whether the entry came from the Sentry SDK itself, was an EF exception message, or was excluded by a log entry filter, so the message was formatted (and allocated) for every entry that was then discarded. The filter check also allocated a closure for every entry through LINQ'sAny(). The formatting now happens after those checks and theAny()is a loop.There are no public API changes.
Issues
No related issue.
Benchmarks
The benchmarks measure, with the real
Huband a no-op transport:ScopeClone:Scope.Clone()of a scope that has one tag and the given number of breadcrumbs.PushScope_NoState:IHub.PushScope()and disposing the returned scope.PushScope_Dictionary8:IHub.PushScope(state)where the state is aDictionary<string, object>with 8 entries, which is whatILogger.BeginScopeis typically given by applications that use structured scopes, and disposing the returned scope.Log_*:SentryLogger.Logfor an entry whose category is from the Sentry SDK, anInformationentry that adds a breadcrumb, and aDebugentry below the configured thresholds.Breadcrumbsis the number of breadcrumbs on the scope that is cloned (the default maximum is 100).ConcurrentDictionaryallocates for each processor, so the figures below are for a machine with 20 logical processors; withDOTNET_PROCESSOR_COUNT=2(a small container) the savings are smaller but still substantial, for examplePushScope_Dictionary8with no breadcrumbs allocates 3.33 KB onmainand 2,056 B on this branch (a ratio of 0.60).The baseline is
mainat f39654d, the commit this branch is based on. Both were run back-to-back in alternating order for two rounds, with--affinity 4095(0xFFF) so that measurements only use the performance cores of the hybrid CPU. The means below are the average of the two rounds. Ratios arebranch / main, so lower is better, and allocation reductions are in bold.Allocations are deterministic, while timings vary from run to run. The
Scopebenchmarks were consistently faster in both rounds (per-round time ratios between 0.11 and 0.66).Log_Information_Breadcrumbis unchanged in time (the two rounds were 1.13 and 0.86), andLog_BelowThreshold_Debugmeasured 4.7 ns against 6.6 ns, which is a difference of about 2 ns on a path that returns before reaching any of the changed code, so I believe it to be noise or code layout.Tests
Sentry.Tests(2,694 passed, 5 skipped) andSentry.Extensions.Logging.Tests(151 passed) pass onnet10.0(Sentry.AspNetCore.Testspassed before the review feedback was addressed, and was not re-run), andSentrybuilds without warnings fornet8.0,net9.0,net10.0,netstandard2.0,netstandard2.1andnet462. The Android, iOS and Mac Catalyst targets were not built locally. New tests cover the lazy collections (including concurrent first access), clones being independent of their source, breadcrumb ordering on clones,ConcurrentQueueLite.EnqueueRange, andSentryLoggernot invoking the formatter for Sentry-originated or below-threshold entries (both of those logger tests fail onmain).ScopeTests.Clone_EveryFieldHandledis updated for the renamed_sdkbacking field.Benchmark code
The
SentryRepoproperty is the root of the clone to benchmark. To comparemainto the branch, build the project twice with-p:SentryRepo=<path to each clone>and-oset to separate folders, withNO_ANDROID=1,NO_IOS=1andNO_MACCATALYST=1set to avoid building the mobile targets, then run each build withdotnet <folder>/sentrybench.dll --filter * --affinity 4095. SetDOTNET_PROCESSOR_COUNT=2to run with 2 logical processors.sentrybench.csprojProgram.cs🤖 Generated with Claude Code