Skip to content

Dataflow analysis lockup compiling System.Runtime.Serialization.Formatters.Tests #73048

Closed
#74111
Closed
Dataflow analysis lockup compiling System.Runtime.Serialization.Formatters.Tests#73048
#74111
Labels
area-NativeAOT-coreclr
Milestone
7.0.0

Description

@MichalStrehovsky
MichalStrehovsky
opened on Jul 29, 2022

It's possible this is an illinker bug, but I'm not ready to spend time extracting an IL Linker repro:

To repro, build the repo how you build it (e.g. build.cmd clr.aot+libs -rc Debug -lc Release). Then:

pushd src\libraries\System.Runtime.Serialization.Formatters\tests
dotnet build -c Release /p:TestNativeAot=true /p:UseNativeAotCoreLib=true /p:RuntimeConfiguration=Debug /p:LibrariesConfiguration=Release /p:TargetFramework=net7.0-windows

Once it tells you it's generating native code, taskkill ilc.exe and find the response file to invoke ilc.exe under artifacts\obj\System.Runtime.Serialization.Formatters.Tests\Release\net7.0-windows\native. Open src\coreclr\tools\aot\ilc.sln in VS, set configuration to the one you built (Debug in my case from the -rc Debug argument), and set the response file as the command line argument. You might want to append --parallelism:1 to make things not multithreaded.

The method we're stuck on is [System.Runtime.Serialization.Formatters.Tests]System.Runtime.Serialization.Formatters.Tests.BinaryFormatterTests.SerializableObjects()

Example callstack:

 	System.Private.CoreLib.dll!System.RuntimeType.GetFieldCandidates(string name, System.Reflection.BindingFlags bindingAttr, bool allowPrefixLookup)	Unknown
 	System.Private.CoreLib.dll!System.RuntimeType.GetFields(System.Reflection.BindingFlags bindingAttr)	Unknown
 	System.Private.CoreLib.dll!System.ValueType.Equals(object obj)	Unknown
 	System.Private.CoreLib.dll!System.Collections.Generic.HashSet<System.Collections.Generic.KeyValuePair<int, ILCompiler.Dataflow.ValueBasicBlockPair>>.Remove(System.Collections.Generic.KeyValuePair<int, ILCompiler.Dataflow.ValueBasicBlockPair> item)	Unknown
 	System.Private.CoreLib.dll!System.Collections.Generic.HashSet<System.Collections.Generic.KeyValuePair<int, ILCompiler.Dataflow.ValueBasicBlockPair>>.ExceptWith(System.Collections.Generic.IEnumerable<System.Collections.Generic.KeyValuePair<int, ILCompiler.Dataflow.ValueBasicBlockPair>> other)	Unknown
>	ILCompiler.Compiler.dll!ILLink.Shared.TrimAnalysis.ArrayValue.Equals(ILLink.Shared.TrimAnalysis.ArrayValue otherArr) Line 79	C#
 	System.Private.CoreLib.dll!System.Collections.Generic.GenericEqualityComparer<ILLink.Shared.DataFlow.SingleValue>.Equals(ILLink.Shared.DataFlow.SingleValue x, ILLink.Shared.DataFlow.SingleValue y)	Unknown
 	System.Private.CoreLib.dll!System.Collections.Generic.HashSet<ILLink.Shared.DataFlow.SingleValue>.AddIfNotPresent(ILLink.Shared.DataFlow.SingleValue value, out int location)	Unknown
 	System.Private.CoreLib.dll!System.Collections.Generic.HashSet<ILLink.Shared.DataFlow.SingleValue>.UnionWith(System.Collections.Generic.IEnumerable<ILLink.Shared.DataFlow.SingleValue> other)	Unknown
 	System.Private.CoreLib.dll!System.Collections.Generic.HashSet<ILLink.Shared.DataFlow.SingleValue>.HashSet(System.Collections.Generic.IEnumerable<ILLink.Shared.DataFlow.SingleValue> collection, System.Collections.Generic.IEqualityComparer<ILLink.Shared.DataFlow.SingleValue> comparer)	Unknown
 	ILCompiler.Compiler.dll!ILLink.Shared.DataFlow.ValueSet<ILLink.Shared.DataFlow.SingleValue>.Clone() Line 190	C#
 	ILCompiler.Compiler.dll!ILLink.Shared.DataFlow.ValueSet<ILLink.Shared.DataFlow.SingleValue>.Meet(ILLink.Shared.DataFlow.ValueSet<ILLink.Shared.DataFlow.SingleValue> left, ILLink.Shared.DataFlow.ValueSet<ILLink.Shared.DataFlow.SingleValue> right) Line 159	C#
 	ILCompiler.Compiler.dll!ILCompiler.Dataflow.MethodBodyScanner.ScanStfld(Internal.IL.MethodIL methodBody, int offset, Internal.IL.ILOpcode opcode, Internal.TypeSystem.FieldDesc field, System.Collections.Generic.Stack<ILCompiler.Dataflow.StackSlot> currentStack, ref ILCompiler.Dataflow.InterproceduralState interproceduralState) Line 1134	C#
 	ILCompiler.Compiler.dll!ILCompiler.Dataflow.MethodBodyScanner.Scan(Internal.IL.MethodIL methodBody, ref ILCompiler.Dataflow.InterproceduralState interproceduralState) Line 664	C#
 	ILCompiler.Compiler.dll!ILCompiler.Dataflow.ReflectionMethodBodyScanner.Scan(Internal.IL.MethodIL methodBody, ref ILCompiler.Dataflow.InterproceduralState interproceduralState) Line 106	C#
 	ILCompiler.Compiler.dll!ILCompiler.Dataflow.MethodBodyScanner.InterproceduralScan(Internal.IL.MethodIL startingMethodBody) Line 317	C#
 	ILCompiler.Compiler.dll!ILCompiler.Dataflow.ReflectionMethodBodyScanner.InterproceduralScan(Internal.IL.MethodIL methodBody) Line 97	C#
 	ILCompiler.Compiler.dll!ILCompiler.Dataflow.ReflectionMethodBodyScanner.ScanAndProcessReturnValue(ILCompiler.DependencyAnalysis.NodeFactory factory, ILLink.Shared.TrimAnalysis.FlowAnnotations annotations, ILCompiler.Logger logger, Internal.IL.MethodIL methodBody) Line 121	C#
 	ILCompiler.Compiler.dll!ILCompiler.DependencyAnalysis.DataflowAnalyzedMethodNode.GetStaticDependencies(ILCompiler.DependencyAnalysis.NodeFactory factory) Line 36	C#

Cc @vitek-karas @sbomer

Activity

  1. MichalStrehovsky
    added
    area-NativeAOT-coreclr
    on Jul 29, 2022
  2. MichalStrehovsky
    added this to the 7.0.0 milestone on Jul 29, 2022

  3. vitek-karas commented on Jul 29, 2022

    @vitek-karas
    vitek-karas
    on Jul 29, 2022
    Member

    I must be doing something wrong - the above repro steps don't hang. ILC successfully process the input in about 20 seconds (maybe a tiny bit more). This is on very recent main branch - no local changes.

  4. MichalStrehovsky commented on Jul 30, 2022

    @MichalStrehovsky
    MichalStrehovsky
    on Jul 30, 2022
    MemberAuthor

    Tried enabling the test in #73104.

    The CI has been locked up like this for 20 minutes.

      System.Runtime.Serialization.Formatters.Tests -> /__w/1/s/artifacts/bin/System.Runtime.Serialization.Formatters.Tests/Release/net7.0-linux/System.Runtime.Serialization.Formatters.Tests.dll
  Optimizing assemblies for size may change the behavior of the app. Be sure to test after publishing. See: https://aka.ms/dotnet-illink
  Generating compatible native code. To optimize for size or speed, visit https://aka.ms/OptimizeNativeAOT

    It's possible the repro steps don't repro it. I'm still trying to figure out what is the right way to dotnet build a single test. What I wrote above has worked for me in the past but maybe it requires some previous artifacts leftover from a previous build attempt. Not sure.

    What I always do locally is I change the definition of SmokeTestProject in src/libraries/tests.proj to only have the test of interest to me and run build.cmd libs.tests -rc Debug -lc Release /p:TestNativeAot=true /p:RunSmokeTestsOnly=true /p:UseNativeAotCoreLib=true. It's not nice but it works :/

  5. MichalStrehovsky
    added a commit that references this issue on Aug 17, 2022

    Add test for dotnet/runtime#73048

    42ab36f

  6. MichalStrehovsky commented on Aug 17, 2022

    @MichalStrehovsky
    MichalStrehovsky
    on Aug 17, 2022
    MemberAuthor

    It's probably not a lockup - I just don't have enough patience. Extracted a repro to dotnet/linker#2977.

  7. sbomer
    added a commit that references this issue on Aug 17, 2022

    Add test for dotnet/runtime#73048 (#2977)

    c42e3cb
  8. MichalStrehovsky
    added a commit that references this issue on Aug 17, 2022

    Don't track current field of state machines

    21423d5
  9. ghost added
    in-prThere is an active PR which will close this issue when it is merged
    on Aug 17, 2022
  10. MichalStrehovsky
    closed this as completedin #74111on Aug 19, 2022
  11. MichalStrehovsky
    added a commit that references this issue on Aug 19, 2022

    Don't track current field of state machines (#74111)

    3932882
  12. ghost removed
    in-prThere is an active PR which will close this issue when it is merged
    on Aug 19, 2022
  13. github-actions
    added a commit that references this issue on Aug 19, 2022

    Don't track current field of state machines

    ace4cd3
  14. jeffschwMSFT
    added a commit that references this issue on Aug 20, 2022

    Don't track current field of state machines (#74216)

    37193fd
  15. ghost locked as resolved and limited conversation to collaborators on Sep 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area-NativeAOT-coreclr

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions