Stuck trying to debug C app crashes on hardware

For the last few months, I've been working on a music player app that uses the Opusfile library to handle decoding .opus audio files, instead of using the system's MP3 decoding API. The app is fairly fleshed out at this point, and while it works great in the Simulator on both macOS and Windows, it crashes immediately at launch on actual Playdate hardware.

I've spent a while trying to isolate the problem, but so far, I haven't been successful. My debug logging indicated that the crash was occurring upon attempting to load a .opus file with Opusfile. I haven't yet received hardware myself, but I've gotten crash logs from several people. However, I haven't been able to use gdb to get line information as described in this thread, as gdb simply tells me that no line number information is available for the addresses from the crash logs. Opusfile and its dependencies are designed to be usable in embedded environments, so I'm not sure what's going on here. The only other thing I can think of is littering the affected Opusfile functions with debug print statements, but that's a time-consuming process both for myself, and for the others who have offered to help me test my app on hardware.

I threw together an example project using Opusfile with the bare minimum amount of code complexity required to reproduce the issue. It simply copies a .opus file into memory, then passes the buffer to Opusfile's op_open_memory function. That function call crashes on hardware, but on the Simulator, the example continues running. It queries the number of samples in the audio file and draws that number to the screen.

I'm hoping someone here is able to identify the problem, perhaps by using a hardware debugger. I've posted the bare minimum example to GitHub. The repository includes everything you need to compile it yourself, but I've included a build here: playdate-opusfile-test.pdx.zip (283.3 KB)

I unfortunately don't have a crash log from the example project, but I have a couple older logs from the full music player:

Logs

(Please note: each test is a different build of my music player.)

Test 1 (from the yellow devkit iMac at the PAX Playdate booth...!)

Test 2

--- crash at 2023/09/18 05:48:00---
build:1fd086bf5715-2.0.3-release.158184-buildbot
   r0:00000000    r1:00000000     r2:00030cb0    r3: 00000000
  r12:006d3000    lr:080353ef     pc:080350fa   psr: 61000000
 cfsr:00010000  hfsr:40000000  mmfar:00000000  bfar: 00000000
rcccsr:00000000
heap allocated: 2639040
Lua totalbytes=391600 GCdebt=-154260 GCestimate=195813 stacksize=80

Test 3

--- crash at 2023/09/29 06:01:38---
build:1fd086bf5715-2.0.3-release.158184-buildbot
   r0:30019b68    r1:00000000     r2:00001000    r3: 00000400
  r12:00000009    lr:24031703     pc:240324ac   psr: 81000200
 cfsr:00010000  hfsr:40000000  mmfar:00000000  bfar: 00000000
rcccsr:00000000
heap allocated: 2639104
Lua totalbytes=391600 GCdebt=-154421 GCestimate=195829 stacksize=80

I'm happy to elaborate on anything, if desired. Thanks for your time.

I've been having the same issue for one of my projects. I get values for lr and pc that are also in the range of 24xxxxxx, which addr2line and gdb say aren't in the range of the executable. My crash logs look like this:

--- crash at 2023/10/14 19:54:22---
build:1fd086bf5715-2.0.3-release.158184-buildbot
   r0:200028e4    r1:30000300     r2:30000300    r3: 300002f8
  r12:a5a5a5a5    lr:2404029d     pc:240310f8   psr: 61000000
 cfsr:00000000  hfsr:40000000  mmfar:00000000  bfar: 00000000
rcccsr:00000000
heap allocated: 7353728
Lua totalbytes=0 GCdebt=0 GCestimate=0 stacksize=0

Do you have any idea when/where these crashes are occurring? Are they consistent?

On a revision B Playdate that just means the crash ocurred in firmware space (which it usually does since the firmware handles errors).

2 Likes

That's good to know. I wish that was better documented. I was going off of File open crashes on device C API - #11 by dave, so I wasn't sure what region those memory addresses were in.

ok, so as per C-based game crashing only on device - #19 by dave, crash logs are currently completely invalid? I really wish this was better documented. The good news is that the crash logs should be fixed in 2.1, which is hopefully coming soon...?

Wanted to post a quick update here:
A couple weeks ago, I figured this out with the help of @joyrider3774. Turns out, the Playdate compiler flags were only being applied to my code, and not any of my libraries. I believe the compiler was producing assembly that was incompatible with the Playdate, causing the app to crash as soon as it tried to call Opusfile functions. I adjusted my CMakeLists.txt to add those flags to all of my subprojects, which fixed the problem.
At some point soon, I'll update the playdate-opusfile-test repo with the working CMakeLists.txt, as I want to turn it into a benchmark so I can try to track down some weird performance discrepancies I've been seeing between Rev A and Rev B consoles. I'll make a new thread for that.

Edit: I've now updated the repository, and started a new thread detailing the performance issues.