Moving notes and progress here.
There are quite a few steps to solving this problem that I’ve clearly identified. Swift as an umbrella project while it looks daunting and huge is very well architected. There are ~4 pieces of the swift project that our attention will be focused on at first:
We need to set some expectations on what is possible with the given hardware and swift. Luckily there is a handful of other open projects attempting to do something similar. Why I added the swift wasm project will be explained in a bit.
But before I go too far into the weeds about these projects and the steps necessary, it should be noted that you can find out for yourself why you might want to use swift on a Playdate. Thanks to swifts excellent C interop and the playdate simulators only real requirement is to be a shared lib with this exact extern C function name and your good, it was relatively painless to get swift working for the simulator so we could get to work building idiomatic swift wrappers over top of the raw C apis. Swift is a fast, low-overhead, and easy to use language. Like rust without all the explicit borrowing or hieroglyphs.
In order to port such a language there is some basically knowledge we need about the device. Luckily, the make or cmake scripts tell you everything you need to know about the processor and includes while the linker script tells you everything you need to know about what the playdate expects and where it should be and what the programs memory may look like.
We care most about these two things since they are what will dictate the steps to how we will need to update swift. We already know swift can work with the playdate library perfectly fine, so none of the other hardware is too relevant (for now). The processor is what we really really care about and it just so happens to be an STM32F7 based on the ARM Cortex-M7 platform. It’s arch is thumbv7em with hardware float support via an FPU. We know swift runs on ARM processors because every single one of their products contains one and well, you can read it in their cmake files a distinct difference though is we are not on the application series of processors, which are more robust and contain features that are geared towards consumer applications. We cannot simply
swiftc With our target and linker settings pointed at our desired arch.
The swift runtime is written in cpp and the stdlib is written in swift. Swift is essentially a thin layer over LLVM and the runtime is compiled with clang++. The first attempt at getting the runtime to compile will be to use clang++ as opposed the the arm distributed gcc compiler since LLVM supports the thumb architecture and most of the tooling is setup around using clang anyway, this is a low friction entry point. Using an external linker which we will likely need to do and the arm gcc objcopy will both work fine with the ELF output by clang, and from what I can tell those are more important to getting something that will execute than the compiler itself. One reason we may not want to do this long term is that the gcc compiler provided does useful things for us, likely has optimizations that LLVM doesn't have, and is officially supported. Swapping out the compiler isn't an impossible task, it is precisely how the
SwiftEmbedded project functions. Which brings me back to the topic of these different existing projects and why we are not using them for anything more than a reference. We will talk about what each project does so there is background.
This is the OG project that brought swift the microcontrollers in the first place. It also comes with an incredible thesis to read. The approach in this project is to allow the compilers driver to be more generic about what sort of tools are passed to it with the idea that it can survive multiple releases of the arm-none-eabi toolchain and require minimal adjustment. This project is also specifically about targeting bare metal completely, i.e. running swift on hardware without any sort of underlying OS. Flash the bin and you are off to the races. It makes changes to the runtime in order to support / stub places where things can't be supported, like signals and threads. It specifically targets the STM32F4 project, but it should work perfectly fine with F7 as well. The way it is built is similar to any other swift toolchain. It has not been updated for 2 years and the toolchain that has been released doesn't properly support Aarch64 based hosts. Bringing this project up to speed with the current swift codebase isn't an easy task because the build scripts have changed radically since 2020 as well as the other tools this relies on.
This project seems to have been started just after the SE project, with the goal of also supporting STM32 and thumb arch. There is idea sharing that has occurred between this and the SE project but it has a very distinct difference from SE, which is that it does not target bare metal OS-less devices. It instead aims to embed with the Zephyr RTOS in order to support the parts of the runtime & stdlib which rely on a posix like environment in order to operate. Studying the project though, I have not really seen where the benefit of having an OS comes in to play yet, as it still stubs out many of the same things that the SE project does. The way this project is constructed is very different from SE as well. It still produces a usable-like-the-others-toolchain, but the way they go about doing that is by creating the typical runtime patches and stdlib patches we see in SE and then avoiding all the work of trying to integrate with the way all other toolchains in the swift project are built by instead using their own scripts to first build the runtime, then build the stdlib, then linking them together, essentially all manually. This is way simpler than attempting to add proper support for a new project to the swift build scripts, but suffers in that there isn't really a long running fork that attempts to track new additions or changes as the come. This makes it more painful to experiment with new features or take advantage of things like the new freestanding stdlib options. It is not really upstreamable, but it does work and they do run a commercial biz on it.
This project ports swift to work with libnx, a user land library for the Nintendo Switch. This project is also based on the work put forth by SE and also attempts to integrate with the swift build system properly. One primary difference is that they technically are targeting a supported processor (aarch64) and it's also an application variant arch. There is slightly less work involved in the port for this, but since the project was hastily put together as the author admits, it does attempt to integrate with the swift build system in a more modern way than the SE project. It suffers, however, from putting a lot of it's modifications in the wrong places instead of trying to work with the build system - completely understandable for a hack, but hard to maintain with an evolving build system. It also has a somewhat posix like environment, so many features that need to be stubbed for other projects don't have to be for this. One extremely cool thing this project does do is integrating libnx as a part of the stdlib. We make a note of this because it makes a ton of sense in the context of building for a highly specialized-not-quite-OS-setting. It does make portability of the swift code slightly harder since you can't simply ignore the fact that there are reliances on libnx that are introduced as a hard dependency. It also uses a clever hack where it pretends to be linux. We will talk about that more later. It has similar stubbing to the other projects, though more minor, and it also attempts to properly integrate with the build system + it is more modern, so the vast changes that have happened to the build system aren't as bad to deal with. I'd also like to note that the changes to swifts build system are very welcome, it may sound negative how I am outlining it, but it's an entirely different topic deserving of a long post in itself.
And now we have circled back just as I promised in paragraph 3. The SwiftWASM project is very interesting. I stumbled upon it after having the shower thought we can see above this post. I very quickly was able to get a WASM runtime stood up and could very easily use the toolchain with no problems at all. It also seemed to support the latest version of swift, which most other projects don't. Stumbling upon this project was a very happy accident for a few reasons:
- it's very piecemeal
- they upstream their work
- they maintain their fork
- they play by the rules (mostly)
If we consider just what exactly it takes to create a WASM fork of swift, we realize we actually share quite a lot in common with the embedded platforms we also desire to target. This project really goes above & beyond in some regards however, such as having essentially customs forks of each part of the swift projects toolchain, all of which they upstream. That might sound crazy but the swift project is actually designed to work like this. Where forked dependencies as you upstream can very simply be killed off once the work is in main. They are also in the unique position of supporting an entirely new triple that is new to most of the projects used by swift, and they have had great success in upstreaming those parts. Luckily, we don't really have that problem with just trying to support a new arch. To recap on what exactly we share:
- One could assume that adding WASM and WASI support is extremely similar to adding support to any sort of new triple and bare metal target, because.. it is! And in fact, during their upstreaming process this was pointed out (todo: add link to PR discussion).
- WASM/WASI doesn't really have an agreed upon way to support many of the same things that bare metal cannot support. Therefore, most of their shims or feature disablings are shared. No thread support, no signal support, no FS support, etc. They do have some amount of posix support, but it's quite minimal.
- One of their end-goals is to support WASM on non-wasi targets too (wasm32-unknown-none). WASI can be considered as a sort of "OS" in some sense even if it is just technically an indirect layer to the actual host OS.
After reading the above, we may have noticed that our goals are very aligned and we actually are better off in that our desired target architecture is already supported by LLVM and doesn't requires tons of upstream change. This is a really important distinction that most of the above projects didn't make. The way you spit out code is distinctly separate from how the runtime works inside of it. The work to support a target that is non-wasi is actually fairly minimal, though they have not explored that path in depth publicly I have run experiments based on a few new additions to the upstream swift project.
(A brief intermission: if it hasn't become obvious yet, my goal here isn't a quick hack to get something to run - we did that with the simulator support and it's not too hard to do what the other projects have done, my goal is longevity so using swift is an actually viable alternative for playdate developers in the long term)
So now we have a background of the existing projects and their strengths / weaknesses and what things we can learn from each. We can now address two particularly hard problems.
Foundation and libdispatch. The former is a collection of APIs that allow for things like interacting with the file system, networking, etc. The APIs contained within Foundation are highly platform specific in many cases. Initial support for Foundation will be non-existent since most of these things have to occur through the SDK API anyway. The latter, libdispatch, is what contains GCD (grand central dispatch) and it is all about threading. We don't support threading, so we will not support libdispatch and Foundation also happens to rely on libdispatch so we tick another box for not supporting Foundation. I think eventually parts of Foundation could be supported, but is out of scope for now.
We are nearing the end of this braindump (I have found that referencing it has been very helpful) so lets list what is next:
- Add a new target architecture
thumbv7em as a supported triple.
- Add bare metal support via
none as an OS.
- Add checks throughout the runtime & stdlib to gate features we can't support
- Create build settings in all the relevant files for configuring a
I will update this post with time, it has been sitting in drafts for far too long!
Edit: A real life ELF file for the correct target!
File created with Hopper 5.5.2
Analysis version 61
32 bits addresses (Little Endian)
I have been beating my head against a wall trying to figure out why my target wouldn't show up and for anyone who comes across this: you need to update the
swift-driver with your OS info, not just the c++ driver