ARM and leg for NativeAOT
After all my travel across NativeAOT land, I became brave. I have one project, where I have Raspberry Pi 3 devices, and which run Blazor Server-side as UI. OS which I using is Raspbian, so no fancy ARM64 support for me. If you don’t know that, NativeAOT has ARM64 support for about half year, I think. But because for some reasons I cannot port my app to ARM64… Actually I software guy, not hardware or kernel guy, so I having issue with sensitivity of GPIO pins on Ubuntu Core, which I cannot solve, but was not present on Raspbian, so I decided to go brute force. Instead of learning hardware, I decide port NativeAOT to ARM.
That sounds scary, but not actually that horrible as somebody would think. Reason for that — reusability! Let’s take a simplistic view on the NativeAOT architecture, and differences between regular .NET.
Disclaimer: Dear reader. Be careful with my explanation, since my understanding is not solid and I may spread misinformation due limited understanding of the system. Consult BOTR and other more authoritative resources. If by a chance you are .NET architect or other .NET developer and find that I’m spreading lies or misinformation, kick my ass and correct me!
NativeAOT share a lot of pieces with regular .NET, but has notable differences. So list of major components:
- Runtime
- Runtime PAL
- IL Compiler (aka ILC)
- Runtime-JIT interface
- Custom System.Private.CoreLib
- JIT
- GC
- Marshalling
- ObjWriter
- Tests
I would try take a look at each piece and note what’s the same and what different between CoreCLR. That maybe do not explain how NativeAOT works as a whole, but at least help better understand that NativeAOT is relatively small bolt-on over CoreCLR
Runtime
This part provides different services to — GC and user code. Functions from these part, are used by all other components and basic blocks for writing correct application. This is where all low-level stuff happens. This part written in C++ and C# and partially in ASM. If you see functions which starts with Rh or Rhp, then you see Runtime function. During platform port, I don’t need to changed that part.
Runtime PAL
This is my own name which I come-up with. This is part of Runtime written in ASM and highly platform specific. Each processor arch has their own folder. Here the minimal list of files which you need for your platform. In my case I just fill missing parts which was not present. A lot of code here similar to code in CoreCLR VM, just with different naming. This is part which you need during poring.
IL Compiler (aka ILC)
This is most important part of NativeAOT, but what’s interesting is that you can think about that as CrossGen2 + some additions, in form of Runtime-JIT interface and Marshalling. This is relatively big difference between CoreCLR and NativeAOT. Only work for porting to other platform related to support of differences between NativeAOT ABI and CoreCLR ABI. I personally was mostly guided by Michal where I should look for fixes. I s
Runtime-JIT interface
This part is interesting in NativeAOT. In order to compile, JIT need to have information about methods and types from runtime. In CoreCLR this information provided by runtime from metadata and probably based on call statistics, but in NativeAOT this information from ILC itself. So all ARM port issues was somewhere around this part.
Custom System.Private.CoreLib
Because we are doing NativeAOT we need tweaks here and there. So that’s why we have custom System.Private.CoreLib. Some parts of new CoreLib is just runtime implemented in C#, as was mentioned previously. Some parts of VM which was written in C++ in CoreCLR moved to C# in NativeAOT, like ComWrappers. And in general there preference to move some parts of runtime to C# in this project. All changes in CoreLib mostly driven by desire have more C# and less C++.
JIT
This component taken as is. Only difference is that it is used at runtime in CoreCLR, but in NativeAOT JIT invoked by ILC to generate code. That code then written using ObjWriter. Want new platform, you better make JIT know about it properties. Also you need write emitter for binary code. This part can be fun.
GC
Most boring part of NativeAOT. It is statically linked to final executable and same happens in CoreCLR. No changes here. New platform — you are on your own. That’s why I play only with existing in CoreCLR platforms.
Marshalling
This is another part of original runtime which was re-written in C# and landed into ILC. Previously it was responsibility of VM generate code for marshalling, not it is ILC responsibility. This part is robust and support a lot of existing marshalling patterns in CoreCLR. There still gaps, but it’s more a matter of real-world need for these cases. Mostly missing support around COM, and IDispatch specifically.
ObjWriter
This part unique to NativeAOT. This is Obj file writer. It uses LLVM to create obj file. You need to generate proper obj files for any new platform.
Tests
This is best part of NativeAOT — test suite from CoreCLR which allow you validate how far you from proper platform support. Without that it would be hard to prove that you have something working in the long run. Thanks for all .NET engineers who creating these property of .NET. Mostly I don’t have a need to create platform specific tests. You probably would be busy with making most tests work before you hit that need.
To summarise, that’s all parts of NativeAOT which may affect portability based on my limited experience working with that project. Now I may tell what you need to do to make ARM application using NativeAOT.
How to build ARM application
In order to build NativeAOT application on ARM32, you need Raspberry Pi and 16Gb flash with installed Raspbian. 16Gb of flash card is absolute minimum, since you need install clang and build NativeAOT on the device. That takes a bit of free space. You will use at least 15Gb, so be careful. If by a chance you would like to run tests suite, you definitely need 32Gb at least.
As a basis you should install prerequisites as written on NativeAOT repo.
Future commands will diverge from regular NativeAOT build, because you have so much free RAM and we build slightly limited version.
Since ARM not yet supported, we have to compile ObjWriter in addition to regular build process.
Then you need to build ObjWriter for NativeAOT using command below.
./build.sh nativeaot.objwriter -c Release
Now build NativeAOT
Release configuration is only one which is working right now. If you build Debug version of NativeAOT runtime, you will hit JIT crash due to differences in CoreCLR and NativeAOT ABI.
If your build halts, you can Ctrl+C and retry command again. I have seen that during first time build. Maybe you can try build with separate command.
./build.sh nativeaot -c Release
./build.sh libs.native+libs.ref -c Release
./build.sh nativeaot+nativeaot.packages -c Release
Release configuration is only one which is working right now. If you build Debug version of NativeAOT runtime, you will hit JIT crash due to differences in CoreCLR and NativeAOT ABI. This issue is tracked in runtime now, and I have hopes that it would be fixed before .NET 6 release.
cd samples/HelloWorld
../../.dotnet/dotnet publish -c Release --self-contained -r linux-arm /p:DisableUnsupportedError=true
You may note that I add --self-contained and /p:DisableUnsupportedError=true . First is to silence warning from .NET SDK, second is to silence error from NativeAOT about unsupported platform.
So far that’s all what you need to make you NativeAOT application on ARM32. I do not have any special application, and did not test beyond HelloWorld, but I think at this point all other BCL and ASP.NET stuff should mostly works. If not, file an issue. I will continue port my app to NativeAOT on ARM, but that’s largerly would depends on ASP.NET Core and Blazor support for NativeAOT. Oops, I mean ILLink.
