Repository navigation
[Perf] Linux/arm64: 1 Regression on 3/23/2026 11:28:12 AM +00:00 #126154
Description
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 22.04 |
| Queue | AmpereUbuntu |
| Baseline | f6d4ffd1dba9478b91d79371e77270d05e2d8a1d |
| Compare | c13de3073c3f8b049fc144393fbce707ca194e2b |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Formats.Tar.Tests.Perf_TarWriter
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 1.90 μs | 2.26 μs | 1.19 | 0.20 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Formats.Tar.Tests.Perf_TarWriter*'Details
System.Formats.Tar.Tests.Perf_TarWriter.PaxTarEntry_WriteEntry
ETL Files
Histogram
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Activity
LoopedBard3 commentedon Mar 26, 2026
🔍 Automated Triage Analysis
Summary: 1. Verification - Run
RunWithBuildAtHash(buildAtHash=["f6d4ffd1dba9478b91d79371e77270d05e2d8a1d", "baa4920c39cbd92a5704218d11a1d6648950b6f5"], testName=["System.Formats.Tar.Tests.Perf_TarWriter.PaxTarEntry_WriteEntry"])to confirm regression at commit baa4920 and quantify exact impact Bisection completed.Finding Confidence: 3/5 (analysis accuracy)
Regression Confidence: 4/5 (likelihood of true regression)
📊 Full Analysis (click to expand)
Issue Overview
Test Analysis
Legend: μ=mean, σ=stddev, n=runs, z=z-score, Thresh%=adaptive threshold
Finding Confidence: 3/5 - Adequate data available with direct commit correlation, but limited by missing historical statistical baseline. Confidence lowered by historical analysis failure but strengthened by precise commit attribution.
Confirmed Regressions
System.Formats.Tar.Tests.Perf_TarWriter.PaxTarEntry_WriteEntry (+19.0%)
PaxTarEntryconstructor to synchronize ExtendedAttributes with public properties, addingSyncStringExtendedAttribute,SyncTimestampExtendedAttribute, andSyncNumericExtendedAttributehelper methods. The benchmark creates a PaxTarEntry with 8 extended attributes (uname, gname, uid, gid, mtime, ctime, atime, MSWINDOWS.rawsd) and immediately writes it. The new synchronization logic added in TarHeader.cs (+124 additions) now processes each extended attribute during construction to keep the dictionary in sync with properties, introducing overhead. The commit added constructor normalization that validates and updates EA entries, which is executed in the hot path. Perfect temporal alignment (committed 2026-03-23 08:21, regression detected 2026-03-23 11:28). Modified files include exact code paths exercised by benchmark: PaxTarEntry.cs, TarHeader.cs, TarHeader.Write.cs. The 19% regression is consistent with added synchronization work per EA entry (8 EAs in benchmark).Open Questions
Recommended Actions
RunWithBuildAtHash(buildAtHash=["f6d4ffd1dba9478b91d79371e77270d05e2d8a1d", "baa4920c39cbd92a5704218d11a1d6648950b6f5"], testName=["System.Formats.Tar.Tests.Perf_TarWriter.PaxTarEntry_WriteEntry"])to confirm regression at commit baa4920 and quantify exact impactSyncStringExtendedAttribute,SyncTimestampExtendedAttribute,SyncNumericExtendedAttribute) to identify optimization opportunities. Focus on whether synchronization work in constructor could be deferred or optimized.collectHardwareCounters=trueto identify whether overhead is CPU-bound (instructions), memory-bound (cache misses), or allocation-relatedStructured Summary (JSON)
{ "schemaVersion": "perf-triage-v1", "issue": 70607, "baselineCommit": "f6d4ffd1dba9478b91d79371e77270d05e2d8a1d", "compareCommit": "c13de3073c3f8b049fc144393fbce707ca194e2b", "commitRangeSize": 24, "historicalContext": { "runsExamined": null, "missing": true, "notes": "Historical analysis tool failed due to SDK protocol version mismatch (SDK expects v2, server reports v3). Falling back to fixed 5% threshold. Statistical baseline (mean, stddev, z-score, adaptive threshold) unavailable." }, "regressions": [ { "test": "System.Formats.Tar.Tests.Perf_TarWriter.PaxTarEntry_WriteEntry", "baseline": 1.9006938505563011, "compare": 2.2633411949115683, "percentDelta": 19.0, "histMean": null, "histStdDev": null, "zScore": null, "isNoise": false, "likelyCommit": "baa4920c39cbd92a5704218d11a1d6648950b6f5", "findingConfidence": 3, "findingConfidenceRationale": "Direct commit correlation with perfect timing and file path match, but capped at 3 due to missing historical statistical baseline for validation", "regressionConfidence": 4, "regressionConfidenceRationale": "Strong signal (19% >> 5% threshold) with surgical commit targeting exact affected code path. Commit adds EA synchronization logic that benchmark exercises. Perfect temporal alignment.", "verified": false, "bisected": false, "bisectionRange": null } ], "nextActions": [ "Verify regression with RunWithBuildAtHash at commit baa4920c vs baseline f6d4ffd1", "Manual code review of TarHeader synchronization helpers for optimization opportunities", "Profile with hardware counters to identify specific bottleneck", "Discuss performance tradeoff with commit authors" ], "generationNotes": "Historical analysis tool unavailable (SDK version mismatch). Analysis based on fixed 5% threshold and commit attribution. Strong candidate identified (baa4920c) with perfect correlation to affected code. Verification recommended to confirm and strengthen confidence to 5/5." }Bisection Results
Total bisection time: 129s
Success rate: 0/1 tests
System.Formats.Tar.Tests.Perf_TarWriter.PaxTarEntry_WriteEntry
Bisection details
nd the added synchronization logic (checking and updating dictionary values on every property setter) would directly impact write performance. The ~13% regression (1992ns → 2259ns) is consistent with the overhead of additional dictionary operations during entry creation and property updates.
Summary: Using historical performance data, I identified that the regression occurred between commits 7c58a24 (good, ~2007ns) and 875672f (bad, ~2259ns). Within this 6-commit range, commit
baa4920c39cbd92a5704218d11a1d6648950b6f5stands out as the clear culprit. This commit adds ExtendedAttributes synchronization to PaxTarEntry property setters - adding 124 lines of synchronization helper code in TarHeader.cs and updating property setters in TarEntry.cs, PaxTarEntry.cs, and PosixTarEntry.cs. The performance test writes PAX tar entries repeatedly, and the new synchronization code (which checks and updates the ExtendedAttributes dictionary on every property set) introduces measurable overhead in the hot path. This is a legitimate functional change that has a direct performance cost.Analysis: This is a true functional regression caused by intentional code changes that add runtime overhead. The commit adds dictionary synchronization operations to maintain API consistency, trading performance for correctness. This is not a code alignment or noise issue - the performance impact is a direct consequence of the new synchronization behavior implemented to fix issue #76405.
Generated by PerfTriageAgent on 2026-03-26 08:53 UTC
dotnet-policy-service commentedon Mar 26, 2026
Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.
DrewScoggins commentedon Mar 26, 2026
Related to #123990
dotnet-policy-service commentedon Mar 26, 2026
Tagging subscribers to this area: @dotnet/area-system-formats-tar
See info in area-owners.md if you want to be subscribed.
LoopedBard3 commentedon Mar 31, 2026
Related Regressions:
Improve PaxTarEntry construction performance (dotnet#132013)
Improve PaxTarEntry construction performance (dotnet#132013)