Skip to content

Skip clearing reference-free HashSet entries - #137594

Closed
tedd wants to merge 2 commits into
dotnet:mainfrom
tedd:codex/hashset-clear-upstream
Closed

tedd wants to merge 2 commits into
dotnet:mainfrom
tedd:codex/hashset-clear-upstream

Conversation

@tedd

@tedd tedd commented Nov 15, 2026

Copy link
Copy Markdown
Contributor

For batches of 16 collections with 65,536 items each, mean Clear() time fell by 73.62-80.76% for int and 92.68-93.39% for a 64-byte value type in two BenchmarkDotNet comparisons.

This copies the behavior already used by HashSet<T>.Remove() and follows #136605.

Clear() now sets the count to zero and resets the buckets and free list without clearing the entry array when T contains no object references. New items overwrite old entries before use. Types that contain references, including structs, are still cleared.

Performance

Selected ClearBench<T>.ClearOnly results for 16 collections per batch. Setup and validation are outside timing. Times are microseconds (us) per batch, shown as mean +/- sample standard deviation for each process.

Type Items Launch Baseline us Patched us Mean time reduction
int 65,536 1 462.966 +/- 159.386 122.119 +/- 36.634 73.62%
int 65,536 2 743.851 +/- 215.651 143.127 +/- 63.925 80.76%
Wide 65,536 1 3289.533 +/- 148.717 240.654 +/- 37.224 92.68%
Wide 65,536 2 3290.321 +/- 126.811 217.613 +/- 34.791 93.39%

BenchmarkDotNet 0.16.0-preview.2, InProcessEmitToolchain, verified High process priority. Each version ran in two fresh processes. Clear-only used 10 warmups and 100 measurements. Clear-and-refill used 8 warmups and 30 measurements targeting 500 ms each.

Windows 11 Pro x64 (build 26200), Ryzen 9 5950X, Release .NET 12.0.0-dev. ReadyToRun and tiered compilation were disabled; concurrent workstation GC was enabled.

Clear-only iterations were shorter than BenchmarkDotNet recommends, and some results had multiple timing clusters. All samples are included. These comparisons do not establish statistical significance. One 65,536-item RefValue clear-and-refill run was 97.9% slower. Three separate process pairs running only this case did not reproduce that increase. The cause remains unknown.

Validation

Full benchmark results and setup

Wide contains eight long fields (64 bytes), compares all fields, and hashes the first. RefValue contains a long and an object reference. Input values are created once and reused.

Benchmark baseline 8ac60f59b95, with this PR's collection implementation applied for the patched runs. Only System.Private.CoreLib.dll differs between the baseline and patched runtime images. Native runtime files are identical.

Baseline/patched order was randomized with seed 136605. No outliers were removed. Clear-only uses one invocation per measurement and checks that all collections are empty afterward.

CycleBench<T>.ClearAndRefill clears and refills one collection per invocation. Refill is timed; existing storage and input values are reused. Each case recorded at least 10 seconds of measured work. Cleanup checks the final count.

Clear-only batches of sixteen collections, in microseconds.

Type Items Launch Baseline us Patched us Mean time reduction
int 1,024 1 5.597 +/- 0.780 2.131 +/- 0.690 61.93%
int 1,024 2 9.066 +/- 5.367 2.279 +/- 0.796 74.86%
Wide 1,024 1 24.171 +/- 9.028 2.651 +/- 0.933 89.03%
Wide 1,024 2 25.953 +/- 12.978 2.665 +/- 1.037 89.73%
RefValue 1,024 1 13.478 +/- 7.812 12.574 +/- 5.784 6.71%
RefValue 1,024 2 12.164 +/- 5.665 12.232 +/- 1.128 -0.56%
int 65,536 1 462.966 +/- 159.386 122.119 +/- 36.634 73.62%
int 65,536 2 743.851 +/- 215.651 143.127 +/- 63.925 80.76%
Wide 65,536 1 3289.533 +/- 148.717 240.654 +/- 37.224 92.68%
Wide 65,536 2 3290.321 +/- 126.811 217.613 +/- 34.791 93.39%
RefValue 65,536 1 1502.826 +/- 241.561 1371.786 +/- 237.687 8.72%
RefValue 65,536 2 1499.497 +/- 431.964 1461.223 +/- 304.780 2.55%

Sustained clear-and-refill cycles, in microseconds.

Type Items Launch Baseline us Patched us Mean time reduction
int 1,024 1 5.863 +/- 0.160 5.669 +/- 0.027 3.31%
int 1,024 2 5.989 +/- 0.179 6.345 +/- 1.167 -5.95%
Wide 1,024 1 13.271 +/- 0.381 11.619 +/- 0.117 12.45%
Wide 1,024 2 13.767 +/- 0.070 13.196 +/- 0.239 4.15%
RefValue 1,024 1 13.593 +/- 0.101 13.505 +/- 0.080 0.64%
RefValue 1,024 2 15.118 +/- 2.975 13.272 +/- 0.064 12.22%
int 65,536 1 377.796 +/- 1.852 364.409 +/- 1.877 3.54%
int 65,536 2 378.927 +/- 3.012 368.656 +/- 8.769 2.71%
Wide 65,536 1 1035.983 +/- 15.044 779.761 +/- 8.994 24.73%
Wide 65,536 2 962.418 +/- 13.702 822.258 +/- 13.571 14.56%
RefValue 65,536 1 1408.155 +/- 8.825 1384.137 +/- 8.467 1.71%
RefValue 65,536 2 1407.672 +/- 30.915 2785.485 +/- 209.541 -97.88%

Higher patched means in at least one process: RefValue at 1,024 items (Clear-only, up to 0.6%), int at 1,024 items (clear-and-refill, up to 6.0%), RefValue at 65,536 items (clear-and-refill, up to 97.9%).

HashSet reference-containing check

In the main matrix, CycleBench<RefValue>.ClearAndRefill at 65,536 items had a 97.9% higher patched mean in launch 2; launch 1 was 1.7% lower. A separate check ran this case alone in three fresh process pairs, without the preceding int and Wide cases. It retained eight warmups, thirty measured iterations, a 500 ms target, High priority, and all observations. Baseline/patched order used seed 136606.

Pair Baseline microseconds Patched microseconds Mean time reduction
1 1950.409 +/- 354.659 1986.600 +/- 296.996 -1.86%
2 2004.222 +/- 206.676 1914.122 +/- 224.338 4.50%
3 1887.303 +/- 168.185 1925.935 +/- 206.007 -2.05%

Values are mean +/- sample SD per process. These isolated comparisons did not reproduce the 97.9% increase, but had large variation within each process. The original observation remains unexplained; the separate check does not rule it out.

Untimed setup diagnostics found identical hash distributions in all six confirmation processes: capacity 75,431, occupied buckets 43,859, maximum bucket load 7, and 28,304 colliding pairs. Those diagnostics were not collected in the original matrix and therefore cannot explain its anomaly.

Timed methods

[Benchmark] public void ClearOnly() => fixture.Clear();

[Benchmark] public void ClearAndRefill()
{
    fixture.Clear();
    fixture.Fill();
}

Note

The implementation, tests, and pull request description were prepared with OpenAI Codex (Fable). I used Tedd.DotNetLab for the performance investigation. I have manually reviewed and checked everything.

@dotnet-policy-service dotnet-policy-service Bot added the community-contribution Indicates that the PR has been added by a community member label Nov 15, 2026
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).
12 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/area-system-collections
See info in area-owners.md if you want to be subscribed.

@jkotas

jkotas commented Nov 15, 2026

Copy link
Copy Markdown
Member

Duplicate of #137582

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-System.Collections community-contribution Indicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants