Nim Bindings vs Lua
I've been trying Nim Bindings lately. Was very interested in the performance comparison of Nim vs Lua. If we assume Nim to be very close to C performance, you can also see this as a rough indication of Lua vs. native performance.
For a fair comparison, I implemented Matt's benchmark as posted in this thread to Nim.
As noted before, there is some variability between runs. Also for the Nim results. For Nim, I would expect timing accuary to account for the variability. I'd say the variability is lower for Nim that for Lua.
I'm using the latest published Nim bindings, and like that @samdze showed me some improved results using experimental compiler optimizations which widen the performance gap between Lua and NIm. His results indicate a performance increase of 7.5x. This is a combination of (unexplained) worse lua performance on his device compared to mine, plus an 1.6x performance increase from the Nim bindings I used to his experimental LTO branch. The results by @samdze can be viewed here
Source (on a branch for another project, sorry for the messy organisation): use C init event for bench · ninovanhooff/Nim-Snake-Playdate@0247678 · GitHub
Here are the results for the current Nim bindings version.
| revA_nim_211_name | revA_nim_211_# | revA_lua_# | revA_lua_211 | |
|---|---|---|---|---|
| nil | 11187 | 3.755287009 | 2979 | nil |
| drawDiagonal | 70 | 0.9210526316 | 76 | drawLine - Diagonal |
| drawHorizontal | 380 | 1.035422343 | 367 | drawLine - Horizontal |
| drawVertical | 82 | 0.9111111111 | 90 | drawLine - Vertical |
| drawRandomDiagonal | 156 | 1.130434783 | 138 | drawLine - Random Diagonal |
| drawLineFillRect | 1567 | 1.748883929 | 896 | drawLine - fillRect |
| drawLineDrawRect | 193 | 1.09039548 | 177 | drawLine - drawRect |
| mathRandomSugar | 1879 | 1.378576669 | 1363 | math.random |
| mathRandomProc | 1668 | 1.319620253 | 1264 | math.random - local |
| mathSin | 10951 | 9.734222222 | 1125 | math.sin |
| mathSinRandom | 1816 | 3.472275335 | 523 | math.sin - random |
| mathCos | 11307 | 9.988515901 | 1132 | math.cos |
| mathCosRandom | 1587 | 2.994339623 | 530 | math.cos - random |
| mathFloor | 11476 | 9.453047776 | 1214 | math.floor - local |
| imageSample - Fast | 10697 | 34.39549839 | 311 | image:sample |
| drawText | 101 | 1.188235294 | 85 | drawText - local |
| drawTextInRect not in C API | 11 | drawTextInRect | ||
| drawRect | 101 | 1.16091954 | 87 | drawRect |
| fillRect | 167 | 1.024539877 | 163 | fillRect |
| drawEllipse | 81 | 1.88372093 | 43 | drawCircleAtPoint |
| fillEllipse | 112 | 2 | 56 | fillCircleAtPoint |
| drawEllipse | 82 | 1.138888889 | 72 | drawCircleInRect |
| fillEllipse | 112 | 1.191489362 | 94 | fillCircleInRect |
| spriteMoveToStatic | 332 | 1.509090909 | 220 | sprite:moveTo - static |
| spriteMoveToRandom | 199 | 1.309210526 | 152 | sprite:moveTo - random |
| spriteSetImage | 5958 | 16.87818697 | 353 | sprite:setImage |
| spriteSetCenterStatic - center not implemented in Nim | 272 | sprite:setCenter - static | ||
| spriteSetCenterToggle - center not implemented in Nim | 254 | sprite:setCenter - toggle | ||
| spriteSetCenterRandom - center not implemented in Nim | 212 | sprite:setCenter - random | ||
| spriteSetZIndex | 9578 | 16.3447099 | 586 | sprite:setZIndex |
| draw | 1178 | 4.252707581 | 277 | image:draw |
| drawLockedLocal - lockFocus not implemented in C | 156 | image:draw - locked | ||
| drawLockedLocal - local is a lua-concept | 151 | image:draw - locked local | ||
| drawPushContext | 1156 | 7.09202454 | 163 | image:draw - pushcontext local |
| imageSample - Slow | 163 | 0.5241157556 | 311 | image:sample |
| average perf increase | ||||
| 4.856087018 |
Observations:
- Are we comparing apples to apples? Looking at the original lua benchmark; Joyrider, Samdze and my results differ significantly. My results, which are used in this comparison, are most favourable to Lua
- All functions except drawLine-vertical are faster for Nim than for Lua. This function is roughly equally as slow on both systems, the fdiiferences are not consistent between runs.
ImageSample - Slowcan be ignored because... - It seems that the C implementation for imageSample is so slow that there are already some optimisations done in Lua. This will have something to do with the generation of BitmapData for every invocation. Still, when working directly with BitmapData, a dramatic increase compared to Lua can still be achieved (34x). If you are making an image editing app, (Playmaker etc.) I would strongly consider C or Nim
- Over all functions, the performance increase averages out to Nim being 5x faster than Lua when ignoring the unoptimized image:sample function
- The performance increase differs greatly per function. For a performance benefit for your project, look at the functions that are used most heavily in your project for every frame. Math is about 10x faster in Nim. When you are using the sprite system for drawing, the performance increase might not be so great (not directly measured by benchmark) when compared to drawing directly to screen (4.3x). If you do a lot of drawing to off-screen images, the performance increase is even more significant, at 7.3x
- Where the Lua sdk is mature and the performance is stabilized, the performance gap will widen because Nim can be tweaked further to be more performant. (5x -> 7.5x as shown by preliminary results)