playdate.geometry.vector2D:normalize() precision

SDK Bug Reports
1

I was reading the accelerometer and calculating the angle to rotate the world in a project. It was working fine but there was a visible snapping issue when I was holding the console nearly vertically.

normalizeBug

After digging into it, it seems there is an issue with playdate.geometry.vector2D:normalize(). If I calculate the normalized vector by hand it fixes the problem.

local vecReference  = playdate.geometry.vector2D.new(0, 1)

local vecCurrentNotNormalized = playdate.geometry.vector2D.new(seqAccelero:Get(), 1)
local vecCurrentLength = vecCurrentNotNormalized:magnitude()

-- normalize
-- local vecCurrent = vecCurrentNotNormalized:normalized()

-- manual
local vecCurrent = playdate.geometry.vector2D.new(vecCurrentNotNormalized.x/vecCurrentLength, vecCurrentNotNormalized.y/vecCurrentLength)

local dp = vecReference:dotProduct(vecCurrent)
local angle = math.acos(dp) * _rad_to_deg * math.sign(vecReference.x * vecCurrent.y - vecReference.y * vecCurrent.x)

normalizeBugFix

2 Likes
2

The problem here is that I thought it would be a good idea to use the fast inverse square root algorithm with only one iteration there, and it's not very accurate around 1.0. Adding a second iteration fixes that big discontinuity, but I'll do some testing a see if there's any real reason not to use sqrtf(). It looks like Cortex-M7 has a built-in sqrt instruction so it's probably plenty fast enough. Targeted for 1.1.1

1 Like