Repository navigation
Improve runtime async throughput and allocation on suspend/resume paths - #136983
stephentoub wants to merge 46 commits into
Conversation
|
Tagging @dotnet/jit-contrib for JIT-EE GUID update |
|
Tagging subscribers to this area: @JulieLeeMSFT, @VSadov |
e4c950a to
155cb0d
Compare
|
I believe this still suffers from #135738, with the linked PR the suspension/resumption can be further improved by a lot. |
|
@stephentoub I know I'm asking for someone else to do work, so feel free to ignore, but once runtime async has fully landed and is enabled everywhere (e.g. in v12) I'd love to see one of your performance retrospectives specifically aimed at it! At the moment as a passive observer it's hard to separate out all the potential benefits/trade-offs etc. From tickets like this it seems a "good but not massive" win, but I'm sure there are cases were it is vastly better than the compiler generated versions |
2ab3ffd to
4f15e23
Compare
d04814e to
b0a4384
Compare
b0a4384 to
cf6fe37
Compare
cf6fe37 to
7ecf232
Compare
2afd36a to
e64e2f8
Compare
Minimal profiling skips count probes for single-block methods, so straight-line methods (notably runtime async variants, which are a single block until await splitting) are instrumented with class probes only. At Tier1 such methods had dynamic PGO data with no counts, which skipped profile synthesis entirely, leaving the root neither trusted nor sufficient and limiting inlining. Synthesize as if there were no data. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Avoids clearing m_stateObject on every resume and re-storing (with a write barrier) the commonly reused continuation on the next suspension. The field is cleared upon completion. VtsInt ~-3%, Direct ~-2%, Pending1 neutral. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…e VTS await metadata in thread state Suspending on an IValueTaskSource no longer writes to a heap object on the suspend path; the source is stored in the stack state and the token/flags/function pointers in the thread's await state. The task reuses the ValueTaskSourceContinuation it already holds, comparing before writing the source, and releases it back to the thread cache at completion or when it next awaits a Task. ~5% faster on repeated IValueTaskSource awaits. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Replace the mutually exclusive CriticalNotifier, Notifier, ValueTaskSource and AwaitedTask fields with a single Notifier object plus a NotifierKind, giving Task.Yield its own kind. This shrinks the stack state that every thunk and dispatcher must zero, removes initialization from Push, and turns HandleSuspended's chain of null checks into a switch. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…meAsyncTask Keeping the continuation installed while dispatching left completed frames' continuations (and their captured state) rooted while callers ran. Clear it when a frame completes or unwinds. Similarly, release a kept-installed ValueTaskSourceContinuation (and its consumed source) when suspending on anything other than another source. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…y 8 bytes Retype Task.m_action as object so that promise-style runtime async tasks can reuse it for their awaited state instead of a dedicated field. Resuming on a custom TaskScheduler now uses a state-based RunOrScheduleAction overload so it doesn't require the cached Action. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Registering a RuntimeAsyncTask as an awaited Task's continuation runs no user code and doesn't consult the thread's contexts, and callers of HandleSuspended restore the thread's contexts afterwards, so the leaf doesn't need to capture them and HandleSuspended doesn't need to restore them. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
When not compiling AOT and no keep-alive is needed, allocate the continuation with the class's standard allocation helper and store Next inline, rather than calling the managed AsyncHelpers.AllocContinuation helper (which goes through an FCALL fast path). Depth10 -8..-13%, YieldOnce -6%, Pending1 -3%. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…yncTask When a non-pattern-matched await (e.g. awaiting a Task stored in a local) needs only to continue on the thread pool, set ContinueOnThreadPool on the suspending frame's continuation instead of storing TaskScheduler.Default as the awaited continuation context. Resumption then skips QueueAwaitContinuationIfNecessary and lets the dispatcher's existing follow-up check handle the hop, as it does for awaits the JIT recognizes. SuppressThrowing awaits keep a transparent marker so failures are still observed. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
When a runtime async frame suspends on a Task and the RuntimeAsyncTask holds no other awaited state, register the continuation directly from DispatchContinuations and FinalizeRuntimeAsyncTask instead of calling HandleSuspended, using a CAS-only fast path for the first continuation of the awaited task. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Emit the thunk's catch(Exception) as an equivalent filter so the JIT can inline it, mark Task-returning thunks FORCEINLINE, let non-async inlinees consume the continuation of their async call, and keep the suspended-path Create* helpers out of line. TopLevel (sync caller awaiting completed RA method) 9.5 -> 7.7 ns. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
When a frame that the dispatcher resumed suspends again, the finish-suspension helper only captures the thread's contexts into the continuation. Flag such continuations and do the capture in DispatchContinuations instead, skipping the helper call in the frame when it has resumed. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Rather than storing a SuppressThrowingAwaitContext sentinel as the awaited task's continuation context, record suppression on the head continuation (new ContinuationFlags.SuppressThrowing bit, carved out of the result index) and handle it in the dispatcher when fetching the awaited task's result. This keeps the completion path free of continuation-context processing. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…edOnTask Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ed task to the dispatcher When an awaited Task completes successfully, the awaiting continuation has no result storage, and it may run inline on the completing thread, the await is already complete. Skip storing the completed task into m_action and having DispatchContinuations consume it, resuming the continuation directly instead. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
When an awaited Task<T> completes successfully and the awaiting continuation can run inline, store the result directly into the continuation and dispatch, rather than stashing the completed task in m_action for the dispatcher. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
When the task already holds a ValueTaskSourceContinuation, the leaf contexts match the thread's and the head continuation carries the await flags, register with the source directly from DispatchContinuations instead of calling HandleSuspended. The OnCompleted shims now catch and route source exceptions to Task.ThrowAsync themselves and report success as a bool. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…s as non-faulting Allows the dead Thread null-check left behind by CaptureContexts/RestoreContexts to be removed from async variants. TopLevel probe: 7.8 -> 5.8 ns. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
When a resumed task suspends again on an UnsafeAwaitAwaiter notifier and its continuation Action is already cached, register it directly rather than going through HandleSuspended's generic setup and switch. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The JIT already skipped the result slot for ignored results of always-suspending helper calls. Extend that to all async calls whose result is unused, and have the resumption stubs skip storing the result when the awaiting continuation has no result storage (null result location), as the runtime helpers already do. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…t-slot handling Capture and restore leaf contexts for Task awaits again so AsyncLocal change notifications fire when a runtime-async method suspends on a Task. Have the interpreter anchor its continuation data on an explicit offset instead of always encoding a result index, so void awaits of Task<T> don't get a result written over saved locals, and tolerate a null result location when resuming interpreted continuations. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Remove the unused dispatcher argument, retain the required function-pointer parameter as a discard, add the missing final newline, and apply the matching Windows/Linux JIT formatter output. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Clear consumed ValueTask sources and isolate completing Task identity while preserving scheduler decisions. Restore tail-await diagnostics and keep adapter exception handlers non-inlineable. Remove incomplete adapter pooling and add lifetime, scheduling, and stack-trace regression coverage. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Preserve captured scheduler and synchronization-context inlining while isolating the completing Task. Cover captured-context restoration with normal and SuppressThrowing awaits, and account for NativeAOT diagnostic formatting without weakening CoreCLR frame checks. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Preserve new ilc diagnostic reports from the NativeAOT_Libraries job in its existing log artifact without changing compiler settings or failure handling. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Restore shared continuation allocation helpers, use normal heuristics for larger async helpers, remove redundant thunk-specific inline restrictions, and avoid requiring NativeAOT diagnostic noise. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
e8fb548 to
ba217a4
Compare
Runtime async already handles synchronous completion efficiently. This PR focuses on suspension and resumption: fewer intermediary objects, smaller continuations, direct result propagation, cheaper exception propagation, and less per-await dispatcher overhead. There is no new public API.
What's in here
Compiler-generated async also uses the changed Task continuation dispatch code. This is a shared path that needs regression coverage, not an exclusively runtime-async change.
Benchmarks
Throughput and NativeAOT size measurements for the current revision are pending.
The published NativeAOT size comparison in MichalStrehovsky/rt-sz#284 measured these whole-PR differences before this round's inlining changes:
In the kestrel-minimal maps, method code attributed to RuntimeAsyncTask grew from 16,168 to 49,889 bytes, including six dispatcher bodies growing from 7,006 to 15,491 bytes combined. The broad forced-inlining annotations have been removed to address that duplication; these snapshot figures do not measure the effect of their removal.
Testing
.\build.cmd clr+libs+host -rc checked -lc releasefailed before the feedback edits at the native CoreCLR prerequisite check because it could not find a Python installation..\build.cmd clr.corelib -rc checked -lc releasepassed after the feedback edits, with zero warnings and errors. This compiles managed CoreLib IL only; native/JIT and test execution remain unverified locally.Generated by Copilot