This is a broader development question. Sorry if this is the wrong forum to ask.
SQLite is useful for many games and apps, and it has fairly small system requirements—“below 300KiB…in minimal stack space (4KiB) and very little heap (100KiB)”. And Playdate has copious RAM.
Has anyone yet attempted to compile SQLite and use it with Playdate’s C SDK? I imagine that the trickiest part might be dealing with filesystem IO (unless its in-memory mode is used) and/or with its OS portability layer / VFS.
It would be nice to see this baked into the SDK.
Out of curiosity, what would you use sql lite for (what type of data)? Do you have a special need for it?
Because typically in game development database are not used except for games that are really data heavy with a lot of cross references (I think football manager is using one for example). This is much simpler and way faster to access information directly in memory instead of querying a database.
Hi! Yes, it’s true that SQLite is less common in games than in other kinds of apps. Just as you say, many games are less “data heavy” than other kinds of apps.
I’m still interested in whether anyone has tried targeting Playdate with SQLite, and I think it would be very interesting if anyone has tried it.
I don’t really want this thread to get too deep into why SQLite might be useful or not useful in games vs. other apps…but I think that SQLite would be useful on Playdate for both non-game apps and data-heavy games.
Playdate doesn’t need to have just games. “Non-game” apps like the Kicooya music player and the Pocket Planner productivity app are more “data heavy” than most games but can still belong on Playdate.
As you know, SQLite is extremely popular in non-game apps due to its benefits for apps: SQLite can make their saved data easier to extend in later versions of the game, encourage self-documenting data schemas, manipulate/filter/transform their data using SQL, automatically cache redundant data in indexes, and read/update stored data faster than with direct filesystem I/O. With SQLite on Playdate, non-game apps on Playdate like Kicooya and Pocket Planner could also get these benefits too.
Some games have complicated data models, like collectible-card games with many decks and card effects or fantasy RPGs with many items spells. As a game’s data model becomes more complicated, it becomes more similar to a mobile app. Many of SQLite’s benefits for apps then become relevant to these data-heavy games.
It’s true that reading/writing data from SQLite is usually not as fast as reading directly from arrays in memory. In an action game, you probably would not want to read and edit data using SQLite in every frame of a realtime action game—instead, you would create some simple data structures and a few progress flags in memory, and you would persist them in a JSON file.
But, data-heavy games like a collectible-card game or fantasy RPG can have complex data models, and they may need to filter, reorder, or cross-reference data frequently. Their data manipulation and storage may be more complicated, more prone to error, and maybe even slower when using arrays/structs and JSON files than with SQLite. In those games, SQLite might be useful for saving game data and manipulating/filtering/transforming game data—just like it is for non-game apps.
It’s true that most games on Playdate have much simpler data models and storing their data in a JSON key–value/tree structure is good enough for them. But there are both non-game apps (like Pocket Planner) and data-heavy games (like RPGs) that have more complicated relationships and incremental, for which SQLite could make development much easier and performance faster.
This might especially be true for games with complex entity–component systems trying to use data-oriented design. People have also pointed at how similar ECS is to databases, too, and SQLite is one of the lightest and most stable databases around. SQLite is still small, pretty fast, simple to use (often simpler than writing a lot of the data-manipulation code yourself), very popular, very stable, and well documented. It’s boring technology in a good way.
So that’s why I’m interested in whether anyone has tried compiling SQLite for Playdate. (I’m exploring making both a productivity app for Playdate—and a turn-based RPG with a complex inventory system and ECS. Both projects would benefit a lot from SQLite.)
Agreed! When I prototyped a small animal crossingish idle game I used SQLite, it made managing all the initialization data so easy. While working I could have the database open in my editor, it fit into my workflow better than a CSV editor would have. It was also helpful in-game because I could print debug info using the SQL functions.
Relevant video:
I achieved this live tweaking of parameters using Nic's Plugin Manager, which allows you to run the game in the Simulator and then build your plugin/parameters file which is then hot reloaded live into the running game. Incudes a demo/example.
I was tweaking values in a Lua table - one click/press to see the changes in the game - but there's nothing stopping you integrating Excel or SQL in your workflow if you so wish.
Nic's Plugin Manager
To answer your original question, I am not aware of any attempt running sqlite on Playdate.
SQLite being quite portable, I would imagine that should be possible to make a version for the Playdate.
I am a proponent that people should use the tool they are comfortable to use and many developers are simply more comfortable and efficient using database because this is a tool they know. So I can see other people benefiting from a port of sqlite.
However databases have some drawbacks, like performance or issues with versioning, that tend to outweigh the benefits in most cases. That's why extremely few developers use databases (in the client of course. server side or for analytics, databases are used a lot). The goal is to have the data in a very simple format that can be loaded fast in memory. Even for a big RPG or card game, loading entries in memory is faster because this type of information take so little space (compared to textures animation, sound, etc.). To edit the data itself however every developer is different, a lot of them like Nintendo uses spreadsheets, other custom software.
Despite that I could imagine that using SQLite to load data in memory (during load times) might be faster than using json.
I'm really curious how this would perform! I doubt I'll have any time in the near future to give it a try, though.. But I do want to note that there's no chance we'd integrate sqlite into the system--we've got nowhere close to the ~300 KB of firmware storage we'd need for it. But compiled into a C game and loaded into the 16MB external ram that'd be no problem!