Skip to content

RegexOptions.Compiled impacts Android startup #71007

Closed
Closed
RegexOptions.Compiled impacts Android startup#71007
Labels
area-System.Text.RegularExpressionsos-android

Description

@jonathanpeppers
jonathanpeppers
opened on Jun 20, 2022

Description

Reviewing a customer issue, we noticed usage of RegexOptions.Compiled impacts startup quite a bit:

image

This was coming from the AppCenter SDK in the customer's app:

https://github.com/microsoft/appcenter-sdk-dotnet/blob/b19ec99e16e554eb1382342b2852b16f7d8f0084/SDK/AppCenter/Microsoft.AppCenter.Shared/AppCenter.cs#L26

This shows up as a regression from Xamarin.Android -> .NET 6, because RegexOptions.Compiled was not implemented at all in mono/mono.

The question here, is how do we get the previous behavior from mono/mono? Is there a way to toggle RuntimeFeature.IsDynamicCodeCompiled?

runtime/src/libraries/System.Text.RegularExpressions/src/System/Text/RegularExpressions/Regex.cs

Line 105 in 807d091

if (RuntimeFeature.IsDynamicCodeCompiled && (options & RegexOptions.Compiled) != 0)

If there is a way to toggle RuntimeFeature.IsDynamicCodeCompiled, should we consider turning on this flag by default on Android? I've found System.Reflection.Emit to be very impactful to startup.

Reproduction Steps

  1. dotnet new android
  2. Open MainActivity.cs and put _ = new Regex(@"([^;=]+)=([^;]+)(?:;\s*)?", RegexOptions.Compiled); in the OnCreate() method.
  3. Build/run with dotnet build -c Release -t:Run

Expected behavior

RegexOptions.Compiled doesn't have a significant impact to Android startup. (Or we have a feature flag to turn it off)

Actual behavior

RegexOptions.Compiled has a significant impact to Android startup.

Regression?

From a customer's perspective they might say this is startup performance regression from Xamarin.Android to .NET 6.

Known Workarounds

Don't use RegexOptions.Compiled, use Normal.

Configuration

I tested this with the .NET 6.0.400-preview.22301.10 SDK, and the SR 1 version of .NET MAUI.

Other information

dotnet trace output: regexoptions.compiled.zip

Activity

  1. ghost added
    area-System.Text.RegularExpressions
    on Jun 20, 2022
  2. ghost added
    untriagedNew issue has not been triaged by the area owner
    on Jun 20, 2022
  4. jonathanpeppers
    moved this to Todo in .NET 7 MAUI Performanceon Jun 20, 2022
  5. jonathanpeppers
    added this to .NET 7 MAUI Performanceon Jun 20, 2022

  6. stephentoub commented on Jun 20, 2022

    @stephentoub
    stephentoub
    on Jun 20, 2022
    Member

    RegexOptions.Compiled is a tradeoff between startup overhead and throughput: it adds overhead to constructing the Regex in exchange for making regexes execute faster. Are you seeing this construction cost increase but throughput not also increase? If they both go up, then the opt-in flag is doing what it's supposed to do, and if a customer doesn't want to tradeoff that construction overhead for throughput, they shouldn't be setting Compiled to opt-in. (And if they want both, they should consider using the regex source generator in .NET 7 instead; this will give them both the construction perf and the throughput perf, at the expense of a bit larger IL size.)

  7. jonathanpeppers commented on Jun 20, 2022

    @jonathanpeppers
    jonathanpeppers
    on Jun 20, 2022
    MemberAuthor

    In this case, the customer's use of RegexOptions.Compiled was coming from a NuGet owned by Microsoft. I was curious if there is some way to opt out of RuntimeFeature.IsDynamicCodeCompiled? Or some way to get back to the behavior they had from Xamarin.Android (mono/mono)?

  8. danmoseley commented on Jun 20, 2022

    @danmoseley
    danmoseley
    on Jun 20, 2022
    Contributor

    Just curious, why do they not pass it when targeting .NET Standard?

    #if NETSTANDARD
        static readonly Regex _secretsRegex = new Regex(SecretsPattern);
#else
        static readonly Regex _secretsRegex = new Regex(SecretsPattern, RegexOptions.Compiled);
#endif

  9. jonathanpeppers commented on Jun 20, 2022

    @jonathanpeppers
    jonathanpeppers
    on Jun 20, 2022
    MemberAuthor

    If you look at blame, it doesn't actually give much detail:

    microsoft/appcenter-sdk-dotnet@70f7944

    I'm not sure what issue they're trying to solve here either. It seems like the old code would have worked in Xamarin.Android in 2019 to me?

  10. danmoseley commented on Jun 20, 2022

    @danmoseley
    danmoseley
    on Jun 20, 2022
    Contributor

    My guess is that they thought it was still broken. I think the to-do here is to just pass on Stephen's guidance. Startup time, throughput, on disk size - pick two!

    If there is a way to toggle RuntimeFeature.IsDynamicCodeCompiled, should we consider turning on this flag by default on Android? I've found System.Reflection.Emit to be very impactful to startup.

    Question for @steveisok I think.

  11. jonathanpeppers commented on Jun 20, 2022

    @jonathanpeppers
    jonathanpeppers
    on Jun 20, 2022
    MemberAuthor

    On Android, we would pretty much never want throughput for anything. So that is why the default setting might be interesting.

  12. danmoseley
    added
    os-android
    on Jun 20, 2022

  14. steveisok commented on Jun 20, 2022

    @steveisok
    steveisok
    on Jun 20, 2022
    Member

    If we were to do anything, I think it would have to be through a runtime feature switch. @jonathanpeppers, how significant are we talking vs what was in mono/mono?

  15. teo-tsirpanis commented on Jun 21, 2022

    @teo-tsirpanis
    teo-tsirpanis
    on Jun 21, 2022
    Contributor

    Just curious, why do they not pass it when targeting .NET Standard?

    @danmoseley they are targeting .NET Standard 1.0 only, which does not have Compiled.

    https://github.com/microsoft/appcenter-sdk-dotnet/blob/b19ec99e16e554eb1382342b2852b16f7d8f0084/SDK/AppCenter/Microsoft.AppCenter/Microsoft.AppCenter.csproj#L6

  16. 2 remaining items

  17. jonathanpeppers
    closed this as completedon Jun 23, 2022
  18. Repository owner moved this from Todo: Performance to Done: Performance in .NET 7 MAUI Performanceon Jun 23, 2022
  19. ghost removed
    untriagedNew issue has not been triaged by the area owner
    on Jun 23, 2022

  20. danmoseley commented on Jun 23, 2022

    @danmoseley
    danmoseley
    on Jun 23, 2022
    Contributor

    Don't use RegexOptions.Compile on mobile, unless you want it for throughput

    Longer term it would be interesting to develop some guidance around use of the regex source generator on mobile. I guess that will emerge as folks use it.

  21. ghost locked as resolved and limited conversation to collaborators on Jul 23, 2022

  22. jonathanpeppers commented on Oct 12, 2022

    @jonathanpeppers
    jonathanpeppers
    on Oct 12, 2022
    MemberAuthor

    It appears the new Regex APIs save about ~93.5ms of startup time on a Pixel 5 in a "hello world" Android app:

    https://github.com/jonathanpeppers/AndroidRegex

    I'm working on a blog post that will mention this in detail.

  23. jonathanpeppers
    moved this from Done: Performance to Done & Blogged in .NET 7 MAUI Performanceon Oct 12, 2022

  24. stephentoub commented on Oct 12, 2022

    @stephentoub
    stephentoub
    on Oct 12, 2022
    Member

    It appears the new Regex APIs save about ~93.5ms of startup time on a Pixel 5 in a "hello world" Android app:

    Nice!

  25. danmoseley commented on Oct 12, 2022

    @danmoseley
    danmoseley
    on Oct 12, 2022
    Contributor

    @jonathanpeppers I see the flame diagram there for "before", I'm curious what the flame diagram looks like "after" (if you collected it)

  26. jonathanpeppers commented on Oct 12, 2022

    @jonathanpeppers
    jonathanpeppers
    on Oct 12, 2022
    MemberAuthor

    Ah yes, so MainActivity..cctor is now so fast it is completely gone from the trace.

    I added the .speedscope/.nettrace files and more screenshots: https://github.com/jonathanpeppers/AndroidRegex

  27. danmoseley commented on Oct 12, 2022

    @danmoseley
    danmoseley
    on Oct 12, 2022
    Contributor

    If I read right, the ctor was 174ms and IsMatch another 49ms. Now the ctor is negligible (because source gen) and IsMatch is only 6ms (because it is smarter about finding there is no possible start position, apparently)

    So I think I'd expect an improvement of over 200ms, not 93.5 ms?
    Also I'm curious what is causing the CompileMethod in the newer IsMatch.

  28. jonathanpeppers commented on Oct 12, 2022

    @jonathanpeppers
    jonathanpeppers
    on Oct 12, 2022
    MemberAuthor

    My general observation, is these timings are a bit slower when traced -- especially on a mobile device running Mono. Maybe @lateralusX knows if there is some amount of overhead?

    I normally test with & without tracing to double-check things.

  29. jonathanpeppers commented on Oct 12, 2022

    @jonathanpeppers
    jonathanpeppers
    on Oct 12, 2022
    MemberAuthor

    Also I'm curious what is causing the CompileMethod in the newer IsMatch.

    RuntimeHelpers.CompileMethod() is a placeholder for time spent in the runtime, so it's likely JIT'ing this method, but it could also be loading additional assemblies, etc.

  30. lateralusX commented on Oct 13, 2022

    @lateralusX
    lateralusX
    on Oct 13, 2022
    Member

    My general observation, is these timings are a bit slower when traced -- especially on a mobile device running Mono. Maybe @lateralusX knows if there is some amount of overhead?

    I normally test with & without tracing to double-check things.

    If you run sampling it will impact process since part of sampling includes a STW (stop the world for every sample) so it will affect timers running longer than the sample interval (1ms be default). If you don't run with sampling, the impact should be minimal. It should also be mentioned that the elapsed time in speed scope is a "calculated metric", but sampling should mainly be used in relation to the total number of samples (percent, flam graphs etc) to spot methods with most samples, that is how tools like perfview visualize sample data. If you want to measure wall clock times, you should do so by add specific trace events, use custom timers or in cases there is support for low level runtime profilers use build in method enter/leave support.

  31. lateralusX commented on Oct 13, 2022

    @lateralusX
    lateralusX
    on Oct 13, 2022
    Member

    Also I'm curious what is causing the CompileMethod in the newer IsMatch.

    RuntimeHelpers.CompileMethod() is a placeholder for time spent in the runtime, so it's likely JIT'ing this method, but it could also be loading additional assemblies, etc.

    Jupp, CompileMethod is a place holder when runtime needs to JIT compile a method. Part of JIT compile a method might results in the need to load additional types (assemblies) and potential do some other runtime work, but CompileMethod gets triggered where there is a need to JIT or load an AOT:ed method. If you run tracing with JIT/Loader events you will find what method gets JIT:ed in those events and if any additional assemblies/types needs to be loaded as part of the process.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area-System.Text.RegularExpressionsos-android

    Type

    No type

    Projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions