[Perf] Linux/x64: Regression in System.Text.Json.Tests.Perf_Basic on 8/19/2024 8:38:43 PM

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	ViperUbuntu
Baseline	ca99ff45874b478c2d7a80bc069a720045579f43
Compare	65141b6f40e4352bfa0da33f0b019c28c1beb28a
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Basic

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
WriteBasicUtf8 - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	341.27 ns	376.39 ns	1.10	0.09	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Basic*'

Details

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: False, DataSize: 10)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@MichalPetryka

Compare range: b1908e2...29f3687
Likely commit: #105098, @MichalPetryka

Related Regressions:
Windows x64: dotnet/perf-autofiling-issues#40240, dotnet/perf-autofiling-issues#40242
Linux Arm64: dotnet/perf-autofiling-issues#40379

@JulieLeeMSFT

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

@EgorBo

@EgorBo, PTAL.

@EgorBot

@EgorBot -intel -amd -commit e96eaee vs previous --filter System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8

@EgorBot

@EgorBot -amd -profiler -commit e96eaee vs previous --envvars DOTNET_JitDisasm:WriteBasicUtf8

// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.
// See the LICENSE file in the project root for more information.

using System.Buffers;
using System.Text.Json.Tests;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;


namespace System.Text.Json.Tests
{
    public class Perf_Basic
    {
        private ArrayBufferWriter<byte> _arrayBufferWriter;
        private int[] _numberArrayValues;
        private byte[] _extraArrayUtf8;
        private byte[] _firstUtf8;
        private byte[] _lastUtf8;
        private byte[] _ageUtf8;
        private byte[] _phoneNumbersUtf8;
        private byte[] _addressUtf8;
        private byte[] _streetUtf8;
        private byte[] _cityUtf8;
        private byte[] _zipUtf8;

        [Params(false)]
        public bool Formatted;

        [Params(false)]
        public bool SkipValidation;

        [Params(10)]
        public int DataSize;

        [GlobalSetup]
        public void Setup()
        {
            _arrayBufferWriter = new ArrayBufferWriter<byte>();

            var random = new Random(42);

            _numberArrayValues = new int[DataSize];

            for (int i = 0; i < DataSize; i++)
            {
                _numberArrayValues[i] = random.Next(-10000, 10000);
            }

            _extraArrayUtf8 = Encoding.UTF8.GetBytes("ExtraArray");
            _firstUtf8 = Encoding.UTF8.GetBytes("first");
            _lastUtf8 = Encoding.UTF8.GetBytes("last");
            _ageUtf8 = Encoding.UTF8.GetBytes("age");
            _phoneNumbersUtf8 = Encoding.UTF8.GetBytes("phoneNumbers");
            _addressUtf8 = Encoding.UTF8.GetBytes("address");
            _streetUtf8 = Encoding.UTF8.GetBytes("street");
            _cityUtf8 = Encoding.UTF8.GetBytes("city");
            _zipUtf8 = Encoding.UTF8.GetBytes("zip");
        }

        [Benchmark]
        [MemoryRandomization]
        public void WriteBasicUtf8()
        {
            _arrayBufferWriter.Clear();

            using (var json = new Utf8JsonWriter(_arrayBufferWriter, new JsonWriterOptions { Indented = Formatted, SkipValidation = SkipValidation }))
            {
                json.WriteStartObject();
                json.WriteNumber(_ageUtf8, 42);
                json.WriteString(_firstUtf8, "John");
                json.WriteString(_lastUtf8, "Smith");
                json.WriteStartArray(_phoneNumbersUtf8);
                json.WriteStringValue("425-000-1212");
                json.WriteStringValue("425-000-1213");
                json.WriteEndArray();
                json.WriteStartObject(_addressUtf8);
                json.WriteString(_streetUtf8, "1 Microsoft Way");
                json.WriteString(_cityUtf8, "Redmond");
                json.WriteNumber(_zipUtf8, 98052);
                json.WriteEndObject();

                json.WriteStartArray(_extraArrayUtf8);
                for (int i = 0; i < DataSize; i++)
                {
                    json.WriteNumberValue(_numberArrayValues[i]);
                }
                json.WriteEndArray();

                json.WriteEndObject();
                json.Flush();
            }
        }
    }
}

seems like a small difference in inlining decision's e.g. TryFormat I don't see anything we can do for 9.0 here.

It also looks like inliner inlined too much in general into WriteBasicUtf8, more than 250 basic blocks, 8kb of code (for both before/after cases)

Navigation Menu

Repository navigation

[Perf] Linux/x64: Regression in System.Text.Json.Tests.Perf_Basic on 8/19/2024 8:38:43 PM #107044

Run Information

Regressions in System.Text.Json.Tests.Perf_Basic

Repro

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: False, DataSize: 10)

ETL Files

Histogram

JIT Disasms

Docs

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[Perf] Linux/x64: Regression in System.Text.Json.Tests.Perf_Basic on 8/19/2024 8:38:43 PM #107044

Description

Run Information

Regressions in System.Text.Json.Tests.Perf_Basic

Repro

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: False, DataSize: 10)

ETL Files

Histogram

JIT Disasms

Docs

Activity

LoopedBard3 commented on Aug 27, 2024

dotnet-policy-service commented on Aug 28, 2024

JulieLeeMSFT commented on Sep 3, 2024

EgorBo commented on Sep 4, 2024

EgorBo commented on Sep 4, 2024

EgorBo commented on Sep 4, 2024

EgorBo commented on Sep 4, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions