Automated testing on PlayDate hardware

Problem

A lot of game developers are still waiting for their PlayDate or cannot afford ordering one just yet. Hardware testing is an essential part of the development process. Inability to quickly test on hardware slows down the game development significantly.

Existing solutions

The only real solution is to post the build on the forum and ask people with PlayDate to try it out.

A big shout-out to people who agree to playtest games on their PlayDate. You are life savers and developers absolutely love you.

Despite being a working system, it is not without drawbacks. There is a limited amount of people who own a PlayDate and only a subset of them can donate their time to monitor the forum and playtest. Playtesting takes time and it is possible for a playtest request to be lost either because it was posted on a busy week or because the playtester is simply not interested.

Proposed solution

Automated testing queue. A solution where developers can submit their .PDX to a queue to be tested on a real PlayDate.

In the simplest setup, it is a single PlayDate on a stand streamed to Twitch, which sequentially runs .PDX files from the queue , allowing each 30 seconds of runtime.

While this solution does not allow developers to test reaction of the game to inputs, it does allow to test various graphical simulations, which often become the limiting factor to the game's performance.

Limiting factors

Currently, there is no way to tell the PlayDate (at least, using public APIs) "hey, drop what you are doing and run this .PDX for 30 seconds". But I am pretty sure that developing such a tool is a realistic goal.

Let me know what do you think, folks.

6 Likes

Currently, there is no way to tell the PlayDate (at least, using public APIs) "hey, drop what you are doing and run this .PDX for 30 seconds". But I am pretty sure that developing such a tool is a realistic goal.

The sdk contains a tool to do this (the "start a new pdx" part) , called 'pdutil'. I use it regularly to send new builds of my game straight to playdate (from the terminal) without going through the simulator

Be aware that using the Mirror app has significant impact on performance though. Gravity Express runs at 30fps on hardware, 22 fps when using Mirror.

IMHO, Panic should set aside 100 playdates for a developer programme. Developers that have a promising project going on here on the forum would get sent one, and they should be passed on to the next dev when the developer has received their own console

1 Like

Did not know about this tool, thank you for sharing!

Actually, I was not even thinking about Mirror - I was going to suggest streaming it through some kind of camera directed to the PlayDate screen :sweat_smile: But if Mirror is hurting performance, maybe streaming through external camera is a preferred way. Actually, we might not need any kind of streaming if PlayDate is able to record a video of its screen and save it somewhere after the experiment .pdx finished.

In any case, I am curious what Panic developers think about it.

1 Like

@Nino Do you know if there is a way to ask Panic to consider such request in a respectful manner?

I do not want to ping random people from Panic who have been kind enough to spend their time to answer some questions on the forum. At the same time, I am not sure how long should we wait before considering this request as "lost from the radars".

It's already been done. There are ~250 developer devices out in the wild as part of the Developer Preview.

I'm not sure how they're managed and can't recall what money was involved to secure one if you were chosen in the draw (there were more applications than available devices).

I do like the "pass it on" idea, as you know. But it requires a lot of management, trust, shipping costs, import duties, etc.

Ah, 250 is quite a lot, time to pass those units on to the next dev then!
Perhaps you could create a post in the Collaboration section, stating that you will take care of shipping and import costs (mention the country where you live for best matching developers).

I still need the one I got through Matt unfortunately, but I should get my own order next month. (you never know with panic timeliness)

Ah, 250 is quite a lot, time to pass those units on to the next dev then!

We had to pay for the devices though. I understand the intent but not everyone can give things away for free and I’m still developing new games.

3 Likes

The cost complicates things, I suppose you need a lot of trust to feel confident that you are going to get your device back. Perhaps you could sell them instead, which would unfortunately drive up the cost for receiving parties. But perhaps there are still some interested devs.

As for still having projects in development, I suggest that those in group 1 and 2 can continue their projects using the production devices they have received in the meantime. Not sure how many have ordered a production unit though.

To not hijack this thread any further, I split of a new one here: Community Developer Programme

I'm attempting to make an emulator in Python 3 (not the Simulator) which I hope would have 100 percent software compatibility with the device. It will mostly be geared toward playing other people's games (since the Simulator is geared towards testing) but I hope for it to work with all device builds of games (both in C and Lua).

Feel free to make an issue on the repo! It's still in its very early stages, so all you can do now is dump a PDX.

https://github.com/scratchminer/pd-emu

2 Likes