C API Sound callbacks happen on another thread?

I am seeing some weird behaviour so I added logging and it seems to confirm that the FilePlayer fadeVolume() function will call its completion callback on a separate thread.

What I see is that the completion callback starts running, and then the system updateCallback runs at the same time, before the completion callback exits.

Is this intended? I see no mention of it in the docs. Are all the sound callbacks called on another thread from the update callback? Do they share a thread?

1 Like

Ah, yep, you're right. That's not good. :confused: I think I was expecting if you're running in C you should be able to handle that on your own, but then we don't give you any tools for the job. I'll try and get this to work like it does in Lua, where the callbacks are deferred to the main run loop, outside of the update callback

okay, now that I'm remembering how this works.. I guess my idea was if the callback is only doing a quick action that won't stall audio rendering that should be fine, and if you need to do more you can set a flag and pick it up in the update callback, which is effectively how it works under the hood on the Lua side. I'd forgotten that we moved audio rendering to a separate task, thought that putting slow code in an audio callback would be dragging the entire system down, but it's not that bad--it'll just make your audio stutter.

I'd like to move those callbacks to the main task like we have in Lua, but I don't think I'll be able to get to it this update. For now I'll make a note in the docs, point out that the callbacks should return quickly to not starve audio

Thanks Dave. It would be very nice to have them run on the same thread interleaved with the update callback.

Hi @dave ,

Is the audio in a separate thread (task?)? I ask because I finally got a version of our C++ game running and the audio (generated by LibXMP) stutters when we run at 30fps. I have to bring the FPS down to 10 to have the audio smooth like butter. This all confuses me, as audio should be on a separate thread/task as the run loop.

Not sure if this helps or not, but our game worked perfectly on the Gen 1 ODROID GO (ESP32 WROVER). :frowning:

Using pd->system->addSource(), here's the AudioHandler function.

int16_t *audioBuffer = nullptr;
size_t audioBufferSize = 0;

int AudioHandler(void *context, int16_t *left, int16_t *right, int len) {

  size_t tmpBufferSize = sizeof(int16_t) * len * 2;
  if (audioBuffer == nullptr) {
    audioBuffer = (int16_t*)malloc(tmpBufferSize);
    audioBufferSize = tmpBufferSize;
  }
  else if (audioBuffer != nullptr && audioBufferSize != tmpBufferSize) {
    audioBuffer = (int16_t*)realloc(audioBuffer, tmpBufferSize);
    audioBufferSize = tmpBufferSize;
  }

  xmp_play_buffer(*currentContext, audioBuffer, (int)tmpBufferSize, 0);
  for (int i = 0; i < len; i++) {
    left[i] = audioBuffer[i * 2];
    right[i] = audioBuffer[i * 2 + 1];
  }

  return 1;
}

Scratch the question @dave =). I should have used printfs to see how things were being called, and yes, AudioHandler is called way faster than Update().

I logged the function calls using pd->system->getCurrentTimeMilliseconds() :slight_smile:

Will look into ways to optimized my code. :nerd_face:

3239 update()
3244 AudioHandler()
3250 AudioHandler()
3256 AudioHandler()
3263 AudioHandler()
3263 update()
3267 AudioHandler()
3273 AudioHandler()
3280 AudioHandler()
3283 update()
3285 AudioHandler()
3290 AudioHandler()
3296 AudioHandler()
3303 AudioHandler()
3304 update()
3308 AudioHandler()
3314 AudioHandler()
3320 AudioHandler()
3323 update()
3325 AudioHandler()
3331 AudioHandler()
3337 AudioHandler()
3343 AudioHandler()
3343 update()

Yes, they're on different tasks, as you discovered, and the audio task runs at higher priority, meaning it'll get called even if your update() function is taking a long time. Currently if your audio callback takes too long and it's still running when the callback for the next block wants to start it'll hang waiting for it to finish and drag down the system. In 2.3 it'll skip that block instead so that things can keep running.

If you haven't already, take a look at the Device Info window while the game is running and check that the CPU is in fact getting overloaded. I assume that's the case here, but if not then I should take a look at what's going on. Another useful tool is the Sampler, which can show you where the code is spending its time. I wonder if libxmp is more efficient if it can render a larger chunk of samples at a time? If that's the case then you could move xmp_play_buffer() to the update function and have it write to a ring buffer that the audio thread reads out of.

I'm not too surprised that the ESP32 performs better here--it's a bit faster than the STM32F7 on coremark tests. We do have an ESP32 in the Playdate but we only use it for Wifi. In retrospect it might have been cool to wire it up so that we could offload audio tasks to it.. but ugh the complexity.

1 Like