[NativeAOT] Simplify build integration with external bundlers for Apple platforms

Description

In order to support targeting Apple platforms NativeAOT build targets need to be integrated with external build systems and bundlers like:

Xamarin.Sdk
AppleAppBuilder

To achieve this, several project properties need to be set in the correct way and we are currently collecting all that is necessary to achieve the integration.

For example: in case of targeting an iOS application with NativeAOT (PublishAot=true) and integrating it with an external build pipeline, these are the properties that need to be set:

<PropertyGroup>
    <!-- This works around an issue in NativeAOT: https://github.com/dotnet/runtime/issues/86186 -->
    <IlcKeepManagedDebuggerSupport>true</IlcKeepManagedDebuggerSupport>
    <!-- We must find the BCL libraries using the runtime pack instead of using the built-in NativeAOT BCL: https://github.com/dotnet/runtime/issues/87060 -->
    <PublishAotUsingRuntimePack>true</PublishAotUsingRuntimePack>
    <!-- Build as a static lib -->
    <NativeLib>static</NativeLib> 
    <!-- Split init for the custom native host -->
    <CustomNativeMain>true</CustomNativeMain>
    <!-- Escape NativeAOT bundling targets -->
    <NativeCompilationDuringPublish>false</NativeCompilationDuringPublish>
    <IlcCompileDependsOn>Compile;ComputeIlcCompileInputs;SetupOSSpecificProps;PrepareForILLink</IlcCompileDependsOn>
</PropertyGroup>
<ItemGroup>
    <DirectPInvoke Include="__Internal" />
</ItemGroup>

NOTE: Please note that the list above is not complete, it merely lists the current workarounds to enable building an iOS application.

To improve the integration process and maintainability it would be beneficial to simplify the integration process when targeting Apple platforms with a single "opt-in" MSBuild property.

I'm not sure I understand the IlcCompileDependsOn and NativeCompilationDuringPublish parts. Is the intention that we would not run the Publish target when building for iOS?

I'm not sure I understand the IlcCompileDependsOn and NativeCompilationDuringPublish parts. Is the intention that we would not run the Publish target when building for iOS?

These are just currently used workarounds to avoid build issues like:

In NativeAOT publish targets: ComputeLinkedFilesToPublish depends on LinkNative, while in iOS setup we would want to do our own native linking, so currently we would just turn it off
The following causes build errors:

runtime/src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Publish.targets

Line 52 in 20ca966

<Error Condition="'$(OutputType)' != 'Library' and '$(NativeLib)' != ''" Text="NativeLib requires OutputType=Library." />
- in the iOS build set up, the OutputType is Exe and we are compiling with <NativeLib>static</NativeLib> (this should probably include an additional condition like: and '$(CustomNativeMain)' != 'true' )
CopyNativeBinary fails as the binary ends up in the .app bundle
etc..

To summarize, I think I was not precise enough in the description of the issue (I will update it), as we are still collecting all that we would need for easier integration and the goal is to hopefully have this all accessible under a single switch.

it would be beneficial to simplify the integration process when targeting Apple platforms with a single "opt-in" MSBuild property.

I do not think that the settings are the only one right way to target Apple platforms. They are specific to Xamarin app-model. Other app models (e.g. Avalonia) may want to do the integration differently.

We try to avoid hardcoding app-model specific settings in the core runtime .target files. For example, we have the same relationship with ASP.NET app models: ASP.NET SDKs have number of custom overrides of the defaults. I think we should follow the same pattern for all appmodels/SDKs, including Xamarin.

It is fine to expose individual features in cleaner way to make the integration easier.

It is fine to expose individual features in cleaner way to make the integration easier.

In particular having to set IlcCompileDependsOn seems like could be implemented in a better way.

IlcKeepManagedDebuggerSupport is fixed by Allow duplicate feature switch settings #87207.
PublishAotUsingRuntimePack will hopefully go away if we move everything to NativeAOT runtime packs ( [NativeAOT] Extend the runtime pack approach to all supported platforms #87060)

I feel the only fragile part is IlcCompileDependsOn, and the main reason why we need to fiddle with it is to allow running both ILLink and ILCompiler. While there are some discussions about avoiding that in the long-term I feel it would be useful to actually fix the scenario to be allowed. That would primarily mean allowing RunILLink=true opt-in here, and implementing the logic to pass ILLink output to ILCompiler on dotnet/runtime or dotnet/sdk side.

[dotnet] Adjust ILC's build dependencies to cope with the fact that I…

In particular having to set IlcCompileDependsOn seems like could be implemented in a better way.

What targets in Microsoft.NETCore.Native.Publish.targets are problematic? We could condition them on some property that the Xamarin props would set so that they don't interfere with how Xamarin wants to set up the flow of inputs/outputs.

The same with LinkNative - I'd suggest looking into why it's even executed when we're not doing a publish. Eventually, we'd want to split LinkNative into two targets - one that constructs the command line arguments we want to pass to the linker (so that the Xamarin targets can pick it up) and another one that actually runs linker (Xamarin would not participate in that).

In particular having to set IlcCompileDependsOn seems like could be implemented in a better way.

What targets in Microsoft.NETCore.Native.Publish.targets are problematic? We could condition them on some property that the Xamarin props would set so that they don't interfere with how Xamarin wants to set up the flow of inputs/outputs.

The same with LinkNative - I'd suggest looking into why it's even executed when we're not doing a publish. Eventually, we'd want to split LinkNative into two targets - one that constructs the command line arguments we want to pass to the linker (so that the Xamarin targets can pick it up) and another one that actually runs linker (Xamarin would not participate in that).

I'd suggest looking into why it's even executed when we're not doing a publish.

Note that we're doing "dotnet publish".

The immediate result I get (with the changes in dotnet/macios#17374) if I set NativeCompilationDuringPublish=true is this:

error MSB4006: There is a circular dependency in the target dependency graph involving target "_ComputeLinkerArguments". Since "IlcCompile" has "DependsOn" dependence on "_ComputeLinkerArguments", the circular is "_ComputeLinkerArguments<-IlcCompile<-LinkNative<-ComputeLinkedFilesToPublish<-ComputeResolvedFilesToPublishList<-_ComputeFrameworkAssemblies<-_ComputeLinkerArguments".

Looks like that's because of:

runtime/src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Publish.targets

Lines 20 to 22 in 5b900dd

    
           <Target Name="ComputeLinkedFilesToPublish" 
        
                   BeforeTargets="ComputeResolvedFilesToPublishList" 
        
                   DependsOnTargets="_ComputeAssembliesToCompileToNative;LinkNative">

where Microsoft.NETCore.Native.Publish.targets injects the ComputeLinkedFilesToPublish into the build chain, and indirectly LinkNative.

So here's problem 1: we don't want to execute NativeAOT's LinkNative target, because we're using the static library ILC creates, and not the final linked output.

Maybe it would make sense to create a property the SDK could set to indicate that we only need the static library, and not the final linked result?

Maybe it would make sense to create a property the SDK could set to indicate that we only need the static library, and not the final linked result?

It is what <NativeLib>Static</NativeLib> does today. It may need some tweaks so that it works well for Xamarin config.

[dotnet] Adjust ILC's build dependencies to cope with the fact that I…

There is more we can do to simplify the integration, so keeping this issue open.

Navigation Menu

Repository navigation

[NativeAOT] Simplify build integration with external bundlers for Apple platforms #87187

Description

4 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	<Target Name="ComputeLinkedFilesToPublish"
	BeforeTargets="ComputeResolvedFilesToPublishList"
	DependsOnTargets="_ComputeAssembliesToCompileToNative;LinkNative">

[NativeAOT] Simplify build integration with external bundlers for Apple platforms #87187

Description

Description

Activity

MichalStrehovsky commented on Jun 7, 2023

ivanpovazan commented on Jun 7, 2023

jkotas commented on Jun 7, 2023

rolfbjarne commented on Jun 7, 2023

filipnavara commented on Jun 8, 2023

4 remaining items

MichalStrehovsky commented on Jun 21, 2023

rolfbjarne commented on Jun 21, 2023

jkotas commented on Jun 21, 2023

kotlarmilos commented on Aug 11, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions