Hi! I've been working on this in my spare time since mid-December and thought I'd write down some of my findings.
Why & what
My background is mostly functional programming in Scala. I've been using the language for over 8 years now, and I feel very confident and productive with it: all the nice features like ADTs, lambdas, a rich collections library, and the convenience of a garbage collector, make it a pleasant experience. I could go on for hours.
When I saw the Playdate, I knew I wanted one - and I was also interested in possibly making games for it - but the SDK choice (C, Lua) didn't satisfy me:
- I find C to be pretty hard to maintain, and very easy to make mistakes with: unions, manual memory management, relatively weak typing, and so on.
- Lua doesn't have static typing (which might be solved by some of its dialects, or type comments), and also I know literally nothing about it (I do know a bit about C from my short time at university). I intend to learn it one day, but I'm hoping this is not the day! (you already know I really like Scala)
Naturally, I wanted to see if I can run Scala on the PD. Well, can I?
Hard facts
Scala's de-facto-default runtime is the JVM. There's no way I can run a whole JVM on a Playdate - and still have cycles left for an actual game - so that's simply out of the question. However, there's this thing called Scala Native that has been in development for almost a decade now, and it's become pretty stable on desktop platforms (x86, aarch64).
So, let's look at today's state of Scala Native and compare it to what the Playdate offers:
SN:
- builds via LLVM/clang
- Supports 64-bit platforms out of the box (Mac, Linux, Windows)
- 32-bit support available in the 0.5 series, which is unstable
- 4 garbage collection modes:
- no GC: pre-allocate a chunk of memory and exit when it's full
- immix (default)
- commix
- boehm (using a dynamically linked libgc)
- requires some C++ libraries, e.g. for exception handling
Playdate:
- 32-bit ARM Cortex-M7
- 16MB memory (might not all be available to games)
- single-threaded game runtime
- limited access to libc (as far as I understand, some things are missing from the toolchain)
- build toolchain is based on GCC. For the simulator, on Mac, I can use clang instead (which makes things easier).
Disclaimer: in case this isn't clear yet, I lived most of my life on the JVM and I'm a native noob. Anything I say about non-JVM things may be complete BS.
The plan
I went through a number of things that failed, but if I were to retroactively write down the plan it would be this:
- Generate API bindings
- Run a Scala game in the simulator
- Run a Scala game on the device, with no GC
- Add GC
- Build a functional API/engine to make it all worth it
1. Generate API bindings
This was pretty easy: I used sn-bindgen to generate the initial bindings, then copy-pasted them to my project. There were two unnamed structs/unions that I had to give names to, but in the end it was a pretty trouble-free process.
We'll get back to this though.
2. Run a Scala game in the simulator
Surprisingly, this was relatively trouble-free part as well. The code is here.
Some important things worth mentioning at this step:
- build target
To build a game for the PD, you have to emit a dynamic library that implements a common interface (the event handler). This means that I can't package my Scala code as an executable - it has to be a library instead.
Thing is, when Scala Native runs as a library, you have to call a special method - ScalaNativeInit() - before making any other Scala calls. That means I still have to wrap my game in some C, even if just to run that function and then proxy to the rest.
So, do I build as a static or dynamic library? In the simulator, we can use shared (dynamically linked) libraries, because it's simply running on our computer's hardware natively (hence it being a simulator and not an emulator). However, the device doesn't currently have a dynamic loader, so I wanted to build static even for the simulator - just to be prepared and prove it'll work.
Thankfully, this is pretty easy with Scala Native:
// in build.sbt
nativeConfig ~= (
_.withBuildTarget(BuildTarget.libraryStatic)
)
I ended up with this C code:
#include "pd_api.h"
#include "demo.h"
int eventHandler(PlaydateAPI *pd, PDSystemEvent event, uint32_t arg)
{
if (event == kEventInit)
{
ScalaNativeInit();
}
return sn_event(pd, event, arg);
}
which calls my exported sn_event function: (full code)
// skipped imports for brevity
object Main {
@exported("sn_event")
def event(
pd: Ptr[PlaydateAPI],
event: PDSystemEvent,
arg: UInt,
): Int = {
val f: CFuncPtr1[Ptr[Byte], CInt] = update
val ptr: Ptr[PDCallbackFunction] = CFuncPtr.toPtr(f).asInstanceOf[Ptr[PDCallbackFunction]]
if (event == kEventInit)
(!(!pd).system).setUpdateCallback(ptr, pd.asInstanceOf[Ptr[Byte]])
0
}
def update(
arg: Ptr[Byte]
): Int = {
val pd = arg.asInstanceOf[Ptr[PlaydateAPI]]
//the usage is actually pretty boring...
1
}
}
And it works!
For the C part of this build, I'm using a custom script - when I was at this stage I didn't grasp CMake/Make that well.
#!/bin/bash
set -euo pipefail
mkdir -p build
mkdir -p build/dep
PLAYDATE_SDK="/Users/kubukoz/Developer/PlaydateSDK"
BASEDIR=$(dirname "$0")
clang -g -g -dynamiclib -rdynamic \
-lm \
-DTARGET_SIMULATOR=1 \
-DTARGET_EXTENSION=1 \
-I . \
-I $PLAYDATE_SDK/C_API \
-I "$BASEDIR/../lib" \
"$BASEDIR/../app/.native/target/scala-3.3.1/libdemo-out.a" \
-Wl,--no-demangle \
-l c++ \
-o "$BASEDIR/build/pdex.dylib" \
"$BASEDIR/src/main.c" \
"$PLAYDATE_SDK/C_API/buildsupport/setup.c"
cp "$BASEDIR/build/pdex.dylib" Source
$PLAYDATE_SDK/bin/pdc "$BASEDIR/Source" "$BASEDIR/HelloWorld.pdx"
open "$BASEDIR/HelloWorld.pdx"
Needless to say, this isn't portable unless you change the PLAYDATE_SDK path to whatever it is for you.
3. Run a Scala game on the device, with no GC
This was much more work. I had to fork scala-native and make quite a bit of changes.
Because there's C++ stuff involved in running Scala Native, I figured I would start with playdate-cpp - it was definitely a useful asset, and to this day I'm relying on it. Hopefully, in the future I can make my build standalone, without having to depend on this - I don't need the full power of C++, most likely just a bunch of stubs will suffice.
Here's a non-exhaustive list of what I've done to make it compile, link and run, in the SN fork itself (diff at the time of writing):
- get rid of some atomic numeric operations (no need for atomics in single-threaded environments)
- add some dummy implementations for things that were needed at link time, but weren't called: this was mostly done by piggybacking on what was done for Windows (based on
#ifdef TARGET_PLAYDATE) - get rid of pwd/cwd, and some more system stuff needed to implement
java.lang.System - hardcode a single thread identifier in the Thread class
- make a NativeThread implementation, so that the main thread can be instantiated. Most methods here are stubbed with System.exit with various codes, so that I'd see which one I need to implement if it happens to be called
- disable delimited continuation support (making them work was way above my skill level and it's just an experimental SN feature I didn't need)
- replace any
fprintf(stderrwith calls to PD's logging function - hardcode the total memory size (relevant for both GC and non-GC work). 16MB was crashing, 14MB seems fine for the time being.
- disable the usage of libunwind (again, way over my skill level at the moment, and I'm fine not having stack traces for now) in functions like
StackTrace_PrintStackTrace - replace
mmap(used for memory allocation in both GC and non-GC) with amalloc. This sounds illegal, but seems to work - replace the uncaught exception handler with something that only prints its message
- hardcode
clock_gettimeto 0
Note: I'm pretty sure some of these things are wrong or unnecessary, but this is not the time to start cleaning up. Well, I did already clean things up a bit, but this is as far as I want to get before the functionality matches my expectations.
More trouble: bindings
Remember how I said bindings worked fine in the simulator? Well, the simulator was running on a 64-bit CPU. Turns out sn-bindgen doesn't support that yet, and it sure as hell doesn't support generating bindings for a platform other than the build platform.
As a workaround, I fixed the relevant places myself (with a bit of help from a fork of sn-bindgen hardcoded to 32 bits, although that wasn't enough). Still, the API seem to crash the game in some ways, so I'm only using some numeric/enum types, and the rest is getting proxied via C.
Back to the hacks
Here's the nativeConfig for building the device-compatible static library:
nativeConfig ~= (
_.withBuildTarget(BuildTarget.libraryStatic)
.withTargetTriple("arm-none-eabi")
.withGC(GC.none)
.withCompileOptions(
Seq(
"-g3",
"-mthumb",
"-mcpu=cortex-m7",
"-mfloat-abi=hard",
"-mfpu=fpv5-sp-d16",
"-D__FPU_USED=1",
"-O2",
"-falign-functions=16",
"-fomit-frame-pointer",
"-gdwarf-2",
"-fverbose-asm",
"-Wdouble-promotion",
"-fno-common",
"-ffunction-sections",
"-fdata-sections",
"-DTARGET_PLAYDATE=1",
"-DTARGET_EXTENSION=1",
"-DDEBUG_PRINT=1",
"-D_LIBCPP_HAS_THREAD_API_PTHREAD=1",
"-MD",
"-MP",
s"-I${playdateSdk / "C_API"}",
"-march=armv7-m",
"-m32",
// "-v",
)
)
.withMultithreadingSupport(false)
)
Again, some of this might be redundant, but it works - with no GC.
Fast forward: current state
In mid-January, I was able to run a game with the logic written fully in Scala - with the C proxies I mentioned above, for the Playdate API calls. You can see a demo video here: Jakub Kozłowski 🐀: "It runs! Scala Native on the Playdate! #playdated…" - Mastodon Party
After that, I was trying to get GC to work. Immix had some trouble, and I considered switching to Boehm (I was told by SN devs that it's simpler), but I didn't have the time to invest into building it for the target platform - and as a statically linked library, no less.
Next steps
I want to go back to some basics: instead of implementing the whole game in Scala, I'll actually minimize that part, and focus on getting Immix to not crash the first time it tries to garbage collect.
In the future, I'd also love to have exception support - but this is not something I urgently need.
I'm also giving a talk about this whole process, as well as the functional game engine, on Scalar (Warsaw, Poland, March 21-22 2024).
