PlaydateUI, a SwiftUI inspired UI framework

Hello!

I have recently been working on something interesting, a sort of SwiftUI but for Playdate! I guess the best name being: PlaydateUI. It's extremely early still and there isn't much code I'd like to share yet. But I can share snippets of how it's used as well as explain a bit of it and keep track of progress in this thread (if this is the correct place).

One of the first problems that lead to building this was simply making the graphics api easier to work with, especially layout. This was a rather easy problem to solve and relies on exposing Yoga to Lua. Yoga is a quick flex box implementation essentially allowing you to build up a view hierarchy and calculate the layout. From there, we can make clever use of tables and classes to build up a pretty nice UI framework. So instead of writing something like this:

-- very simple example
function playdate.update()
  gfx.clear()
  gfx.drawText("Hello", 10, 10)
  gfx.drawText("World", 10, 20)
end

You can instead write code like this:

View('ContentView')
function ContentView:body()
  return VStack(leading) {
    Text("Hello"),
    Text("World"),
    padding = 10
  }
end

function playdate.update()
  mount(ContentView)
end

Static rendering is pretty simple and works very well now, with most layout modifiers supported, as seen with padding above. There are also the expected layout primitives HStack, ZStack, and many other primitives like Circle, RoundedRectangle, etc. I have not yet started working on things like color.

Where things get really interesting though is when you start involving state.

View('CountView', { count = 0 })
function CountView:body()
  if playdate.buttonJustReleased(playdate.kButtonA) then
    self.count = self.count + 1
  end

  if playdate.buttonJustReleased(playdate.kButtonB) then
    self.count = self.count - 1
  end

  return VStack() {
    Text("Count: "..self.count),
    Text("Count x2: "..self.count * 2)
  }
end

function playdate.update()
  mount(CountView)
end

This is the part that has the least polish currently. I plan to build a "focus engine" that will allow for views to be able to be "focused" on with the d-pad and have actions tied directly to the view itself as opposed to listening on the loop as we can see above. One can imagine this would work very similarly to SwiftUI apps for Apple TV.

Probably the more interesting thing though is how the above approach looks sort of like magic. While this portion is far from complete and literally only a prototype of a prototype- you will notice that we must be initializing views in order to compose them, so the logic follows that self.count should always be reset to 0. But... its not.

The way this bit of magic works is that during the execution phase the renderer walks through the view tree and grabbing any table values (like self.count) that aren't the known functions implemented for views and storing them in a long lived table after a render is completed. Before that, the values are looked up and injected into the view. This lets us "pretend" to have long living state while still having this nice composition we see. You could have multiple CountViews and they each would have their own state. I have not exactly worked out how to achieve this uniqueness yet of course.

This is all very inefficient currently, my understanding of Lua is new (a couple days old) and the entire view tree is recreated on each update and plenty of stuff is only barely working. I hope folks find it interesting and I would love to collaborate on what the design should look like, though I am pretty sure this is NOT idiomatic Lua :laughing:

As I get closer to something releasable I will share here & on GitHub. Would love to hear thoughts or anything you may want to see.

16 Likes

I guess it would be helpful to attach something! Here is a random example showing state, circle primitive, and hstack in action.
Untitled

1 Like

That's awesome! I'm not familiar with SwiftUI but this resemble any good UI framework I've used before. I'm also not super versed in Lua, so I won't be able to help too much on that but I do have some suggestions.

I think one thing that could be interesting to take into account right from the start is to make it as modular as possible as the paradigm of having big libraries doing everything on a machine with such limited resources could be problematic.

As for what I'd like to see, a strong parenting and animation system also having some sort of 9 slice utility could be nice and helpful for UI building.

In any case this is a great initiative and I'd love to contribute when you make it open source!

2 Likes

Would love to hear more about "modular from the start", as I think I am keeping it that way. All the layout primitives (3 of them) provided should be more than enough to create any layout you want. And as for the other primitives like text, circle, rectangle, etc. they are mostly really thin wrappers around drawing calls to the playdate graphics lib. So, supporting animator & 9 slice should be completely possible. I'm also curious what you mean by "parenting" I am unfamiliar with that, do you mean uni-directional data flow? If so, that too, is supported, though Lua lets you abuse this, and I'd suggest people don't do that because eventually the idea is more concentrated update calls based on state changes or inputs.

Thanks for the kind words! Its inching along. I'd open source it now but I am changing stuff so rapidly it would probably not be worth following code wise :laughing: once I figure out state management then I will probably open source, as that is the trickiest bit currently, and is taking a lot of iteration.

edit: would like to add that modifiers are also nothing special - they don't work like SwiftUI though, and I am not sure if they will. They behave more like props you pass to a view ala react, so they are essentially just "reserved" keywords for a view.

One solution to the state problem that I also considered was using a specific project layout approach for the way that components are created. Each component being their own file, with the body being the lone function in that file, then walking the tree and executing render on each. This would allow for state to be held locally to the component file. One problem with doing that though is there is no way (I think?) to inspect the local state of a component from global. By managing all of state we can create optimizations in the future around choosing when to redraw and you also are able to create multiple components in a single file and compose those together.

My current working solution is the first arg of each view is a uid which is used to identify it's state container, each place the view is used you use a new uid. The goal now is to remove the need for the developer to create the uid themselves :smiley:

Things like environment will also be supported and are a rather simple concept, as it's basically just global DI into views.
The concept of preferences can sort of be handled by abusing/using shared variables.

First of all - this is amazing :heart:
I've been doing swiftui full time in the last half a year (after about 9 years of UIKit) and it really is insanely powerful once you get past the shift in thinking... and it's funny how playing with playdate sdk was such a "back to basics" compared not just to swiftui but also uikit... crazy that you got such a lean api working already :slight_smile:

Re state handling - maybe for a simple system like playdate it make sense to take inspiration from redux in some way and use global state? Unless you think it will ruin the ability for doing smart optimization about re-draw only what necessary?

Re the comment about "modular from the start" - I agree it probably makes sense to make this like lego pieces, on a system like playdate every kb might matter... so for example I should be able to pick & choose to depend on (and pay the memory cost of) things like animation support, environment support, each type of view independently, etc.

for uid - swiftui uses the full "chain" of the view in the tree from the root view (unless of course overriding with the .id(..) modifier...)

2 Likes

This looks awesome! I can't wait to see where you take it.

1 Like

SwiftUI is my day job as well!

definitely want to stick with the state injection method, it will make things like bindings and draw-on-state change possible. will also be able to easily inject environment through similar mechanism.

I got state working + the uid problem solved. Essentially uid for now is just an increment of the view count. Since views should never be "removed" the counts will remain stable. Not a super great solution since we are keeping stuff in memory we don't need to of course - it's going to be like that regardless I think thought, since conditional statements are going to be functions (boo).

I guess I see what folks mean by modular. Currently, the library is less than half the size of the graphics library, with the only two primitives being text and rectangle (rewrote a bunch and didn't port the other ones yet). The growth will be fairly linear as I wrap again, but finite once all primitives are wrapped. I guess what is weird to me is you don't just import part of the graphics lib. I can see animation being separate, but that will depend on how they actually work within the framework too, and I haven't thought much about it.

Speaking of having to create wrappers for primitives, I may well be able to not create wrappers at all and instead switch over the primitive, this would also cut down on memory overhead, since each rect/circle/etc wouldn't need to hold a drawing function.

environment is basically free :stuck_out_tongue: it's just a hash that gets injected on render essentially.

I don't have the hardware yet so I have no clue what the performance on this compares like.

1 Like

I can test on device.

Really interested in this. Very cool

1 Like

If anyone has the chance, I would love if you could run the attached PDX on your hardware. There is some wonky stuff around exceptions and linking - so I ended up adding a small hack to the ld map, curious if it works or just crashes:
uiLibrary.pdx.zip (231.8 KB)

The hack being that I needed to stub end in the text section of link_map.ld. I am unsure if how I did that was a good idea :laughing: and doubly unsure as to why I need to do this. I also added -specs=nano.specs -specs=nosys.specs to the linker options which cleaned up most of the exception related linking problems.

Hm, turns out that was not the solution. Only crashes. Would love input on using cpp with this thing.

Edit: ended up going back to the C route (though I don't think it was necessary, now) because the real issue was the library I'm using referencing malloc.

Updated version using the proper memory functions, anyone with device - please test!
uiLibrary.pdx.zip (64.9 KB)

WOOP!

Works on device.

1 Like

Yes!!! Thank you so much. Drawing time isn't quite as bad as I thought it would be either. Not sure if I built for release or debug, prob debug too. Should be a bit better. Awesome!

1 Like

how does the perf feel? it doesn't really do much yet. but I am working on it!

1 Like

label updates seem "instant"

Yeah, so long as the draw time is less than 100ms it shouldn't really be noticeable. Will need to generate a very complex layout to see how well it holds up in those conditions. Memory pressure and cpu usage seem minimal on the simulator.

Since this is working on device, I feel a bit more comfortable sharing the repo. It comes with no instructions or help or anything else currently and has no guarantee of api stability.

Edit: based on those numbers, it's about 10x slower than the simulator :laughing:

1 Like

Would anyone with a device mind testing out the attached pdx? Attempting to move back to cpp again.
PlaydateUI.pdx.zip (324.2 KB)

(I seriously can't wait to get my pd, batch 2!)

Basic image support!

1 Like