How to judge performance without hardware?

SDK Get Help
1

Are there any benchmarks available for comparing simulator to hardware performance? I've noted the API docs state the simulator will run faster and may not be indicative of hardware performance, but seeing as it will be over a year until I can get my hands on an actual Playdate, benchmarking without hardware seems vital for developers like myself.

2

It is very difficult to properly judge or guarantee how a game would run on the real hardware. In my experience the console can handle pretty well most games as long as you don't go too crazy with the number of moving elements (like tons of particles or enemies), rotate and scale images, or heavy calculation (like physics, any simulation, fancy rendering etc.)

If you would like to see your game running on the real hardware, you can always ask on the forum. People who have the console can record a video to show you how it is running.

2 Likes
3

Maybe it's a good idea we can start a thread offering to run games? So all the requests are in one place?

4 Likes
4

And there we can add some info upfront to make it more effective like showing the framerate and describing which part should be tested.

2 Likes
5

I think it would be cool if the simulator was able to "throttle" its execution or throw warnings when exceeding known limits (memory, etc) of the hardware. Of course, its a simulator, not an emulator, so I am unsure of how possible that would be.

6

On the Lua side I added a time penalty where it counts the number of cycles through the VM each update then adds count * 0.3 uS to the frame delay, based on my tests on the hardware where I maxed out at around 3.3M ops/s. That's not going to be an accurate representation at all of performance on the hardware, but it'll hopefully stop someone from writing a huge Lua framework that runs okay in the simulator but gets 1 frame a second on the hardware. That's the only place I can think of to add a bottleneck like that.. :confused:

As far as memory constraints, you can turn on Malloc Pool in the simulator so that it uses the same allocator as the hardware on a 16MB chunk of memory. This won't mirror device usage exactly because the simulator uses 64 bit pointers instead of 32, but at least it's erring on the safe side there.

1 Like
7

I'd be curious what you mean on the Lua side, I understand the words but not how to do it :laughing:

8

Dave means this mechanism is already in place for Lua code running on the Simulator

9

I guess I am asking what he means by "adds a time penalty" and where and how.

10

In the update loop in the simulator. It calls the update function in the core code shared by the device and the simulators then it does all the OS-y type stuff like maintaining the update timer to match the display rate set in code. We already do some funky timing like this if you've set the display refresh rate to 0--instead of a constant frame delay, it pauses for n/240 * 0.02s, where n is the number of rows that were updated, which is how long it takes the hardware to update a row on the display. (Which is how this display updates, a row at a time.)

2 Likes
11

This reminds me, I have noticed some odd simulator behavior in relation to marking rows updated using the C sdk. Setup is using C update callback, the display refresh rate is set to 0 and you are directly manipulating the frame buffer (via getFrame), then calling drawFPS after.

When using markUpdatedRows there is a particular difference in performance between how you mark those rows as updated.

For instance, if you use markUpdatedRows(0, 239) then the simulator will correctly show 50 for the FPS.

But if you are manipulating the framebuffer in a loop, calling markUpdatedRows(y, y) for each line before drawFPS gets called or the update callback is returned then FPS will hover around 99.

This makes me wonder, what does markUpdatedRows actually do behind the scenes... and is this a bug that is somehow getting around the funky timing limit?

Edit: here is an example of what I mean, this code runs in the simulator much faster than if we marked all the rows as updated outside of the loop (even if all the rows inside the loop end up being marked as updated)

int SDL_PLAYDATE_UpdateWindowFramebuffer(_THIS, SDL_Window * window, const SDL_Rect * rects, int numrects)
 {
     if (!window->surface) {
         return SDL_SetError("Couldn't find surface for window");
     }

     Uint32* pixels = window->surface->pixels;
     Uint8* frame = pd->graphics->getFrame();

     for (int i = 0;i<number_of_bytes;i++) {
         SDL_PD_Pixel pixel = SDL_pd_pixels[i];
         Uint8 d_row = bayer2_rows[pixel.y];
         Uint8 d_col = bayer2_cols[pixel.x];

         Uint8 r; Uint8 g; Uint8 b;
         SDL_GetRGB(pixels[i], window->surface->format, &r, &g, &b);
         Uint8 intensity = r_d * r + g_d * g + b_d * b;
         Uint8 previous = frame[pixel.i];
         if (intensity < bayer2[d_col][d_row]) {
             frame[pixel.i] &= pixel.black_mask;
         } else {
             frame[pixel.i] |= pixel.white_mask;
         }
         int isDirty = previous != frame[pixel.i];
         if (isDirty) {
             pd->graphics->markUpdatedRows(pixel.y, pixel.y);
         }
     }

     return 0;
 }
12

I believe @Dave has said there are some delays in the simulator for screen updates. It could be that some paths have been missed. It could also be that I'm misremembering.

13

Turns out the Windows and Linux sims aren't doing the line-by-line timing for refresh rate 0, maybe that's the issue? I believe we'll have that fixed in the next release.

2 Likes
14

I am on Mac fwiw, I kind of assumed that it pauses every time you call markUpdatedRows as opposed to collecting a count of rows & then pausing on the update callback or some such :grimacing: