Skip to content

Zero-initialize the return-value range in the Leave2 slow path - #135449

Merged
lateralusX merged 1 commit into
dotnet:mainfrom
pcshrosbree:dev/profileleave-leave2-range-init
Oct 19, 2026
Merged

lateralusX merged 1 commit into
dotnet:mainfrom
pcshrosbree:dev/profileleave-leave2-range-init

Conversation

@pcshrosbree

Copy link
Copy Markdown
Contributor

ProfileLeave's Leave2 slow path declares COR_PRF_FUNCTION_ARGUMENT_RANGE argumentRange; uninitialized, calls ProfilingGetFunctionLeave3Info, checks the result only with _ASSERTE(hr == S_OK), and passes &argumentRange to the profiler's FunctionLeave2 hook.

The query does not always write the range. The slow path is taken whenever COR_PRF_ENABLE_FRAME_INFO, COR_PRF_ENABLE_STACK_SNAPSHOT or COR_PRF_ENABLE_FUNCTION_RETVAL is set, but the query fills the range only when COR_PRF_ENABLE_FUNCTION_RETVAL is set. It also returns E_UNEXPECTED before writing anything if its iterator allocation fails. In those cases the hook receives whatever was on the stack. This zero-initializes the range, so the hook gets an empty one.

I measured the first case with a FunctionLeave2 hook that enables COR_PRF_ENABLE_FRAME_INFO without COR_PRF_ENABLE_FUNCTION_RETVAL and reads only the range's two fields (Release builds; the Leave2 path there is identical to main's):

  • linux-x64: before this change about 85% of Leave2 calls got a non-empty range (2730 of 3226 in one run, similar in two more), with lengths around 2 GB and stack addresses as the start. With it, 0 of about 3300 in each of three runs. Reverting brought the garbage back.
  • Windows x64: 109,031 non-empty ranges out of 113,064 Leave2 calls before, and 0 of 113,072 after.

The FunctionLeave2 docs say return value information requires COR_PRF_ENABLE_FUNCTION_RETVAL, so a profiler following them doesn't read the range in that configuration. Still, the hook is handed a pointer to an indeterminate struct. The suspension-Leave discussion on #131985 is also considering CORPROF_E_DATAINCOMPLETE for this query, which would add another non-filling path (#131985 (comment)).

Testing: the experiment above. On linux-x64 Release, the src/tests/profiler tree passes with this change (together with an unrelated AMD64 profiler fix) on main as of dca76ff; the four tests that skip themselves unless tiered compilation is disabled were run with DOTNET_TieredCompilation=0. The profiler code involved is unchanged between dca76ff and this branch's base. The allocation-failure path was not exercised. Tailcall2 has no range. Enter2 passes NULL argument info when arguments are off, but its allocated argument buffer is left unfilled if the query's own allocation fails; I haven't changed or tested that here.

ProfileLeave's Leave2 slow path passes argumentRange to the profiler's FunctionLeave2 hook,
but ProfilingGetFunctionLeave3Info does not always write it. The slow path is taken whenever
COR_PRF_ENABLE_FRAME_INFO, COR_PRF_ENABLE_STACK_SNAPSHOT or COR_PRF_ENABLE_FUNCTION_RETVAL is
set, while the query fills the range only when COR_PRF_ENABLE_FUNCTION_RETVAL is set; it also
returns early, before filling it, on an allocation failure. Only a debug assert checks the
result. In those cases the hook received indeterminate values from the stack.

Initialize the range so the hook gets an empty range instead.
@dotnet-policy-service dotnet-policy-service Bot added the community-contribution Indicates that the PR has been added by a community member label Oct 8, 2026
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @agocke
See info in area-owners.md if you want to be subscribed.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @steveisok, @tommcdon, @dotnet/dotnet-diag
See info in area-owners.md if you want to be subscribed.

@lateralusX
lateralusX merged commit e3b77ef into dotnet:main Oct 19, 2026
132 checks passed
@pcshrosbree

Copy link
Copy Markdown
Contributor Author

Thank you for the review and the merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-Diagnostics-coreclr community-contribution Indicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants