Skip to content

Instantly share code, notes, and snippets.

@JohnnyonFlame

JohnnyonFlame/README.md

Last active May 28, 2024 11:41
Show Gist options
  • Save JohnnyonFlame/63e2ae902329ace9a828d2eeaec88eba to your computer and use it in GitHub Desktop.
Save JohnnyonFlame/63e2ae902329ace9a828d2eeaec88eba to your computer and use it in GitHub Desktop.
Download ZIP
Building NativeAOT games for old ARM Devices and outdated software.
Raw
README.md

DOTNET Runtime for old GLIBC (ArmHF support included)!

Our problem here is that there are classes of devices that are stuck on very old glibc versions (e.g. 2.26) and armhf.

We're solving this by compiling the .NET 9 artifacts with an Ubuntu bionic (18.04) rootfs, this should be close enough to just work. At the time of writing this, we had only .NET 9 preview 4 with armhf support, so that's what we're using (tag: v9.0.0-preview.4.24266.19, you can change this if necessary).

sudo apt-get install build-essential liblttng-ust-dev git cmake ninja-build pkg-config clang qemu-user qemu-user-static binfmt-support debootstrap
sudo apt-get install crossbuild-essential-armhf crossbuild-essential-arm64 binutils-arm-linux-gnueabihf binutils-arm-linux-gnueabi binutils-aarch64-linux-gnu llvm
git clone https://github.com/dotnet/runtime -b v9.0.0-preview.4.24266.19 dotnet-runtime-v9p4 --depth 1
cd dotnet-runtime-v9p4
sudo ./eng/common/cross/build-rootfs.sh arm bionic
sudo ./eng/common/cross/build-rootfs.sh arm64 bionic
ROOTFS_DIR=`pwd`/.tools/rootfs/arm ./src/coreclr/build-runtime.sh -arm -hostarch x64 -component crosscomponents --cmakeargs "-DCLR_CROSS_COMPONENTS_BUILD=1" --configuration Release
ROOTFS_DIR=`pwd`/.tools/rootfs/arm ./build.sh clr         --cross --arch arm --cmakeargs "-DCLR_ARM_FPU_CAPABILITY=0x3" --cmakeargs "-DCLR_ARM_FPU_TYPE=vfpv3" --librariesConfiguration Release --configuration Release --runtimeConfiguration Release
ROOTFS_DIR=`pwd`/.tools/rootfs/arm ./build.sh libs        --cross --arch arm --cmakeargs "-DCLR_ARM_FPU_CAPABILITY=0x3" --cmakeargs "-DCLR_ARM_FPU_TYPE=vfpv3" --librariesConfiguration Release --configuration Release --runtimeConfiguration Release
ROOTFS_DIR=`pwd`/.tools/rootfs/arm ./build.sh host        --cross --arch arm --cmakeargs "-DCLR_ARM_FPU_CAPABILITY=0x3" --cmakeargs "-DCLR_ARM_FPU_TYPE=vfpv3" --librariesConfiguration Release --configuration Release --runtimeConfiguration Release
ROOTFS_DIR=`pwd`/.tools/rootfs/arm64 ./src/coreclr/build-runtime.sh -arm64 -hostarch x64 -component crosscomponents --cmakeargs "-DCLR_CROSS_COMPONENTS_BUILD=1" --configuration Release
ROOTFS_DIR=`pwd`/.tools/rootfs/arm64 ./build.sh clr         --cross --arch arm64 --librariesConfiguration Release --configuration Release --runtimeConfiguration Release
ROOTFS_DIR=`pwd`/.tools/rootfs/arm64 ./build.sh libs        --cross --arch arm64 --librariesConfiguration Release --configuration Release --runtimeConfiguration Release
ROOTFS_DIR=`pwd`/.tools/rootfs/arm64 ./build.sh host        --cross --arch arm64 --librariesConfiguration Release --configuration Release --runtimeConfiguration Release

Converting to .NET Core 9

With this, you should have all the artifacts you need to build and deploy NativeAOT for the target platform.

Now configure your application to use netcoreapp9.0 as a target:

  <PropertyGroup>
    <AssemblyName>Wizorb</AssemblyName>
    <GenerateAssemblyInfo>False</GenerateAssemblyInfo>
    <OutputType>WinExe</OutputType>
    <SelfContained>true</SelfContained>
    <TargetFramework>netcoreapp9.0</TargetFramework>
    <PlatformTarget>anycpu</PlatformTarget>
    <PublishAot>true</PublishAot>
  </PropertyGroup>

There are plenty of other little differences you might want to resolve, such as updating from NewtonSoft's JSON:

https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json/migrate-from-newtonsoft

A good tip here is for you to learn how to use the Source Generators, e.g.:

using System.Text.Json.Serialization;
using System.Collections.Generic;
using Mogade.JsonConverters;

namespace Mogade
{
    [JsonSerializable(typeof(SavedScore))]
    [JsonSerializable(typeof(LeaderboardHighScoreEntry[]))]
    [JsonSerializable(typeof(List<string>))]
    [JsonSerializable(typeof(Response<SavedScore>))]
    public partial class MogadeJsonContext : JsonSerializerContext
    {
    }
}

Now we can fix out types:

- 				r.Data = JsonSerializer.Deserialize<ICollection<string>>(r.Raw);
+ 				r.Data = JsonSerializer.Deserialize(r.Raw, MogadeJsonContext.Default.Achievement);
- 				r.Data = JsonSerializer.Deserialize<ICollection<string>>(r.Raw);
+ 				r.Data = JsonSerializer.Deserialize(r.Raw, MogadeJsonContext.Default.ListString);

Here's an example of a WriteJson method one of the JsonConverter had, you may want to convert it to Write.

-	public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
+	public override void Write(Utf8JsonWriter writer, IDictionary<string, int> value, JsonSerializerOptions options)

Fixing NativeAOT incompatibilities

A common issue you'll have is with types being trimmed out of your binary. One solution for this is setting up a Runtime Directives file (rd.xml, see: https://learn.microsoft.com/en-us/windows/uwp/dotnet-native/runtime-directives-rd-xml-configuration-file-reference)

On your .csproj add this definition:

  <ItemGroup>
    <RdXmlFile Include="rd.xml" />
  </ItemGroup>

Here's an example of Runtime Directives for a non-trivial game. Take notice of the instantiated concrete types we define here.

These are necessary if you run into errors such as "ContentLoadException: Could not find ContentTypeReader Type" even after adding your entire assemblies into it, such as the case with ParisEngine not being enough for the Animation/Frame/PropertyData types to be available.

    <Assembly Name="Wizorb" Dynamic="Required All" />
    <Assembly Name="ParisEngine" Dynamic="Required All">
    </Assembly>
    <Assembly Name="EasyPak" Dynamic="Required All" />
    <Assembly Name="mscorlib" />
    <Assembly Name="FNA" Dynamic="Required All">
        <Type Name="Microsoft.Xna.Framework.Content.ListReader`1[[Paris.Engine.Graphics.Animations.AnimatedObject2dData+Animation, ParisEngine]]" Dynamic="Required All" />
        <Type Name="Microsoft.Xna.Framework.Content.ListReader`1[[Paris.Engine.Graphics.Animations.AnimatedObject2dData+Frame, ParisEngine]]" Dynamic="Required All" />
        <Type Name="Microsoft.Xna.Framework.Content.ListReader`1[[Paris.Engine.PropertyData, ParisEngine]]" Dynamic="Required All" />
        <Type Name="Microsoft.Xna.Framework.Content.ListReader`1[[System.String, mscorlib]]" Dynamic="Required All" />
    </Assembly>

Building the actual application:

After all of the .NET Core and NativeAOT incompatibilities are fixed, we want to build our application using our SysRoot and Artifacts we compiled previously:

dotnet publish -r linux-arm64 -c Release \
	-p:SysRoot=$(realpath ~/dotnet-runtime-v9p4/.tools/rootfs/arm64) \
	-p:IlcSdkPath=$(realpath ~/dotnet-runtime-v9p4/artifacts/bin/coreclr/linux.arm64.Release/aotsdk)/ \
	-p:IlcFrameworkNativePath=$(realpath ~/dotnet-runtime-v9p4/artifacts/bin/runtime/net9.0-linux-Release-arm64/)/

dotnet publish -r linux-arm -c Release \
	-p:SysRoot=$(realpath ~/dotnet-runtime-v9p4/.tools/rootfs/arm) \
	-p:IlcSdkPath=$(realpath ~/dotnet-runtime-v9p4/artifacts/bin/coreclr/linux.arm.Release/aotsdk)/ \
	-p:IlcFrameworkNativePath=$(realpath ~/dotnet-runtime-v9p4/artifacts/bin/runtime/net9.0-linux-Release-arm/)/ \
	-p:LinkerFlavor=lld

If everything went accordingly, you should have a NativeAOT build of your game for deployment.

How about FNALibs?

Setup your CMake toolchain file ~/arm-dotnet.cmake, remember to set the appropriate CMAKE_SYSTEM_PROCESSOR for your board.

set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_PROCESSOR arm)
set(TARGET_ABI "linux-gnueabihf")

SET(CMAKE_C_COMPILER   ${CMAKE_SYSTEM_PROCESSOR}-${TARGET_ABI}-gcc)
SET(CMAKE_CXX_COMPILER ${CMAKE_SYSTEM_PROCESSOR}-${TARGET_ABI}-g++)

SET(CMAKE_SYSROOT ~/dotnet-runtime-v9p4/.tools/rootfs/arm)
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)

SET(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)

find_program(GCC_FULL_PATH ${CMAKE_SYSTEM_PROCESSOR}-${TARGET_ABI}-gcc)
if (NOT GCC_FULL_PATH)
  message(FATAL_ERROR "Cross-compiler ${CMAKE_SYSTEM_PROCESSOR}-${TARGET_ABI}-gcc not found")
endif ()
get_filename_component(GCC_DIR ${GCC_FULL_PATH} PATH)
SET(CMAKE_LINKER       ${GCC_DIR}/${CMAKE_SYSTEM_PROCESSOR}-${TARGET_ABI}-ld      CACHE FILEPATH "linker")
SET(CMAKE_ASM_COMPILER ${GCC_DIR}/${CMAKE_SYSTEM_PROCESSOR}-${TARGET_ABI}-as      CACHE FILEPATH "assembler")
SET(CMAKE_OBJCOPY      ${GCC_DIR}/${CMAKE_SYSTEM_PROCESSOR}-${TARGET_ABI}-objcopy CACHE FILEPATH "objcopy")
SET(CMAKE_STRIP        ${GCC_DIR}/${CMAKE_SYSTEM_PROCESSOR}-${TARGET_ABI}-strip   CACHE FILEPATH "strip")
SET(CMAKE_CPP          ${GCC_DIR}/${CMAKE_SYSTEM_PROCESSOR}-${TARGET_ABI}-cpp     CACHE FILEPATH "cpp")
set(PKG_CONFIG_EXECUTABLE "/usr/bin/pkg-config")
set(PKG_CONFIG_PATH "${CMAKE_SYSROOT}/usr/lib/arm-linux-gnueabihf/pkgconfig")
set(CMAKE_SYSTEM_PROCESSOR cortex-a7)

We need to build FNAlibs for our game. As an example, we're building the armhf targets, first let's build a dummy SDL2 for the includes and shared libraries.

You are only using this to build FNAlibs, you must provide your own SDL2 for the target platform with whatever customizations you need (e.g. DispmanX, mali-fbdev, etc).

git clone https://github.com/libsdl-org/SDL --depth 1 -b release-2.28.1 sdl2.28.1
cd sdl2.28.1
export SDL_INSTALLED_PREFIX=$(pwd)/installed
cmake . -B build-arm -DCMAKE_TOOLCHAIN_FILE=~/armhf.cmake -DCMAKE_INSTALL_PREFIX="$SDL_INSTALLED_PREFIX"
make -C build-arm -j8 install

Now navigate to your FNA folder on your project, and let's build FNA3D and FAudio:

cd lib/FNA3D
cmake . -B build-arm -DCMAKE_TOOLCHAIN_FILE=~/armhf.cmake -DCMAKE_INSTALL_PREFIX="$SDL_INSTALLED_PREFIX" -DCMAKE_BUILD_TYPE=MinSizeRel
make -j16 -C build-arm
cd lib/FAudio
cmake . -B build-arm -DCMAKE_TOOLCHAIN_FILE=~/armhf.cmake -DCMAKE_INSTALL_PREFIX="$SDL_INSTALLED_PREFIX" -DCMAKE_BUILD_TYPE=MinSizeRel
make -j16 -C build-arm

Tada! You now have everything you should need to deploy your application. You can fetch the ICU libraries from the sysroot rootfs we generated on the start of this document.

Here's an example of a deployed and fully populated game directory:

./libs.arm
./libs.arm/libFAudio.so
./libs.arm/libFNA3D.so
./libs.arm/libicudata.so.60
./libs.arm/libicui18n.so.60
./libs.arm/libicuio.so.60
./libs.arm/libiculx.so.60
./libs.arm/libicutest.so.60
./libs.arm/libicutu.so.60
./libs.arm/libicuuc.so.60
./Content
./Content/<...>
./Wizorb.arm
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment