Add .NET 10 support via NativeAOT-LLVM#4741
Draft
JasonAtClockwork wants to merge 21 commits intomasterfrom
Draft
Add .NET 10 support via NativeAOT-LLVM#4741JasonAtClockwork wants to merge 21 commits intomasterfrom
JasonAtClockwork wants to merge 21 commits intomasterfrom
Conversation
The experimental NativeAOT-LLVM build path (EXPERIMENTAL_WASM_AOT=1) was missing several host function imports added in ABI versions 10.0-10.4, and the compiler package reference was hardcoded to Windows x64 only. Changes to SpacetimeDB.Runtime.targets: - Add missing spacetime_10.0 imports: datastore_update_bsatn, identity - Add all spacetime_10.1 imports: bytes_source_remaining_length - Add all spacetime_10.2 imports: get_jwt - Add all spacetime_10.3 imports: procedure_start_mut_tx, procedure_commit_mut_tx, procedure_abort_mut_tx, procedure_http_request - Add all spacetime_10.4 imports: datastore_index_scan_point_bsatn, datastore_delete_by_index_scan_point_bsatn - Replace hardcoded runtime.win-x64 package reference with runtime.$(NETCoreSdkPortableRuntimeIdentifier) so the AOT compiler package resolves correctly on both Windows x64 and Linux x64 - Use explicit version strings instead of $(SpacetimeNamespace) variable Changes to ci.yml: - Add AOT build smoketest in csharp-testsuite job to verify the NativeAOT-LLVM build path works on Linux x64 See #4514 for full context on the C# AOT situation.
#4601) # Description of Changes * Add dotnet-experimental feed + package source mapping for LLVM packages in `sdks/csharp/tools~/write-nuget-config.sh`, so generated NuGet.Config files include NativeAOT-LLVM prerequisites. * Make LLVM toolchain packages explicit dependencies in `SpacetimeDB.Runtime` to ensure restores succeed even when LLVM dependencies are only referenced through the `.nupkg`. * Import the LLVM targets from the package when `EXPERIMENTAL_WASM_AOT=1` to enable NativeAOT build steps without relying on downstream package reference resolution. # Context Changes are required to get `NativeAOT-LLVM` in #4515 to build correct, but moving the packages closer to the build, to ensure they get into the Nuget restore successfully. Additional changes where needed to `write-nuget-config.sh` to allow `Nuget.Config` files generated with required changes during regression testing. # API and ABI breaking changes None. # Expected complexity level and risk 2 (Low–moderate). Changes are scoped to build/restore infrastructure and package configuration. # Testing - [X] Built CLI locally - [X] Ran `run-regression-tests.sh` without errors --------- Co-authored-by: Jason Larabie <jason@clockworklabs.io>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com> Signed-off-by: Ryan <r.ekhoff@clockworklabs.io>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com> Signed-off-by: Ryan <r.ekhoff@clockworklabs.io>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com> Signed-off-by: Ryan <r.ekhoff@clockworklabs.io>
…f/update-nativeaot-llvm-infrastructure
…odule-path` is used
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of changes
.NET 8.NET 10behindEXPERIMENTAL_WASM_AOT=1SpacetimeDB.Runtimedefaults tonet8.0and switches tonet10.0for NativeAOTSpacetimeDB.BSATN.Runtimedefaults tonetstandard2.1;net8.0and switches tonetstandard2.1;net10.0.NET 8-safe shim forWasmImportLinkageAttributeand annotated the NativeAOT host imports that require itMicrosoft.DotNet.ILCompiler.LLVMruntime.$(NETCoreSdkPortableRuntimeIdentifier).Microsoft.DotNet.ILCompiler.LLVMnet8.0net10.0whenEXPERIMENTAL_WASM_AOT=1.witfiles fromIlcFrameworkNativePathIlcSdkPathspacetime init --native-aotfor C# so it injects the full known-good.NET 10NativeAOT project shape intoStdbModule.csprojspacetime publish --native-aotso the C# build path:.NET 10NativeAOT output layoutwasm32-unknown-wasip1Why the heavy changes?
NativeAOT-LLVM currently does not cleanly support the exact output shape we need for SpacetimeDB: a WASI Preview 1/core wasm module that can be hosted by SpacetimeDB/wasmtime today.
In practice, the LLVM/WASI toolchain wants to flow component-model metadata (
*.wit) into the link step, which causes linker arguments like--component-typeand breaks production of the core module we actually need. The important upstream issue is:[NativeAOT-LLVM] Output non componentdotnet/runtimelab#3144The current workaround is to overlay
IlcFrameworkNativePathand exclude*.witfiles beforeLinkNativeLlvm, while preservingIlcSdkPath. Through testing, this was the only project-level workaround that proved necessary; the earlierIlcLlvmTargetproperty andWasmComponentTypeWit Remove=...item removal were not required once the overlay was in place.In order to strip out the
*.witfiles it was necessary to expand the module.csprojto include the following, which will copy out the outputs without the.witfiles allowing the build to continue in a core module format:Why
.NET 10And Not.NET 9I also investigated
.NET 9NativeAOT-LLVM as part of this work, but it is blocked by an upstreamMemoryMarshaltype-load failure in the NativeAOT-LLVM toolchain:browser-wasm - System.TypeLoadException: Could not load type 'System.Runtime.InteropServices.MemoryMarshal'browser-wasm - System.TypeLoadException: Could not load type 'System.Runtime.InteropServices.MemoryMarshal' dotnet/runtimelab#2731That issue reproduced in our C# module runtime path as well, so this PR keeps the stable path on
.NET 8and targets the experimental NativeAOT path on.NET 10.API and ABI breaking changes
.csprojto disable the .NET 10 specific requirements.Expected complexity level and risk
2 - The biggest risk is not the stable
.NET 8path, but future changes in the upstream NativeAOT-LLVM/WASI pipeline. This PR intentionally keeps the experimental logic explicit in the generated module project rather than hiding it in transitive package magic, because that proved more reliable during restore/publish testing. We also discussed if we run into troubles due to future changes from theNativeAOT-LLVMbranch to pin our version which can be done through thePackageReferences.Testing
.NET 8path.NET 10NativeAOT pathspacetime init --lang csharp --native-aotgenerates a workingStdbModule.csprojspacetime publish --native-aotsuccessfully builds and publishes a generated C# project