fillPolygon crashes on some tiny polygons

I have a game that resizes a bunch of polygons as the user turns the crank. When some of these polygons get really small the device crashes. I distilled one of the polygons down to the minimum data that still crashes the simulator on my intel iMac.

To reproduce, just run this sample main.lua:

import "CoreLibs/graphics"

function playdate.update()
    local poly = playdate.geometry.polygon.new(
        138.42, 232.48, 138.52, 232.62, 138.375, 232.71, 138.365, 232.72, 138.5, 232.845, 138.48, 232.885, 138.485, 232.905, 137.885, 233.505
    )
    poly:close()
    playdate.graphics.fillPolygon(poly)
end

Here's the exception from the error report:

{"codes":"0x0000000000000001, 0x0000000100000004","rawCodes":[1,4294967300],"type":"EXC_BAD_ACCESS","signal":"SIGSEGV","subtype":"KERN_INVALID_ADDRESS at 0x0000000100000004"},

I can workaround the issue by checking the width and height of the polygon's bounds and skipping the fillPolygon() call when the bounds are smaller than 2 pixels, but thought it was worth reporting.

1 Like

Thanks for the report! I saw that one pop up on memfault yesterday. Oddly the numbers in the core dump didn't cause a crash for me, but I noticed the polygon had a bunch of vertices packed together in a small box so made a quick fuzzer that tried random variations on that and quickly found a crasher. I see what the problem is: There's a case where I'm miscounting the max. number of active edges to allocate for the scanline algorithm, so it overruns the buffer later on. I'll fix that and also add a bounds check so it doesn't crash if I've missed another case where this happens.

1 Like