I put together a tutorial for the new custom synth generator feature in 1.12 (didn't wind up in 1.11 after all like I'd tweeted above) as I was working on it, figured I'd go ahead and post it here for feedback.
I've structured this as if you're wrapping third party code into a custom generator so I've kept the synth implementation separate from the Playdate code. In this example we've got a very simple 2-operator FM synth, no real features yet, just to make sure synth->setGenerator() works and we can get sound through. The important bit here is the render callback which fills the audio buffers passed to it, nsamples (=256) samples at a time. Since we called setGenerator with the stereo flag equal to zero, only the left buffer is passed to the render function.
static int fm_render(void* userdata, int32_t* left, int32_t* right, int nsamples, uint32_t rate, int32_t drate)
{
FMSynth* fm = userdata;
for ( int i = 0; i < nsamples; ++i )
left[i] = FMSynth_getNextSample(fm) * 0x7fffff;
return nsamples;
}
Samples in the output buffer are in signed 8.24 fixed-point format, meaning +/- 0x7fffff is the normal range and values outside this range will be clipped, unless they're scaled down, e.g. by lowering the channel volume. We ignore rate and drate here because we're using the pitch from the noteOn callback. If we wanted to add a frequency modulator by calling snd->synth->setFrequencyModulator() we'd need to use rate and drate instead for it to work correctly. (We'll do that in a future installment to show how that works.)
I've gone back and forth on whether to stuff more comments in the code.. I don't usually comment my code much because it inevitably winds up out of date (cf the bag of waffles in my freezer labeled "pork chops") and also because my code is so perfect and well-written it transcends the need for comments. So I'm opting for more description in the posts here instead.
Let me know if there's anything particularly confusing here, anything you'd like me to expand on, or if you have any other comments or suggestions. Future installments:
adding Lua
parameters and waveform plotting
implementing setParameter so you can plug signals/modulators in
custom signals
optimize for device
try to emulate the DX-7
Taking the kid to the pool now, will post more tonight
Okay, so let's get a basic Lua interface around this thing to make it easier to play around with new features. Here's what we're going for:
s = playdate.sound.synth.newFM()
s:playNote("C4", 1.0, 2.0)
We'll add a single newFM() function to the system playdate.sound.synth class that returns a new synth with our FM generator set on it. It's just a normal playdate.sound.synth as far as the runtime is concerned, so the existing playNote() function works as expected, we can use our new synth in playdate.sound.instrument, etc.
(Huh. There's a small change in the synth implementation, looks like I fixed the way operator feedback works. Since we aren't using feedback yet anyway I'll probably just take that out of parts 1 and 2..)
So now we can play our synth from Lua, awesome! The obvious next step is to add accessors for the various parameters so we can tweak how it sounds beyond just the pitch. But here we've got a small problem: Ideally we'd make a subclass of playdate.sound.synth so we can add functions to change our parameters on this type of synth, but the Playdate runtime doesn't yet give you a good way to do that. This seems like it might work:
but unfortunately Playdate's Lua interface doesn't currently check the metatable chain to identify types--meaning if you pass one of these FMSynth objects to a Playdate function expecting a playdate.sound.synth it won't work. There are two solutions for this, each with their own drawbacks, which we'll explore in upcoming parts.
The first thing we'll do this round is add setters for the various parameters in our FM synth so we can make some real noise: level, feedback, and ratio for both the modulator and the carrier. (To do: find/write a good explanation of what these parameters do. The hits I get on google for "fm synthesis" are okay, but usually written for a specific program.) On the Lua side we have that subclass problem mentioned previously, so here's how we'll deal with it this time: In addition to returning a playdate.sound.synth object that uses our custom generator we'll also return a separate control object for doing things specific to our FM synth--that is, what we'd normally put in a subclass.
synth, fm = snd.synth.newFM() -- both control the same synth :(
Here we have synth to do the normal synth stuff like playing notes and setting envelope parameters, and fm to tweak those FM parameters we added. We'll use the x/y axes of the accelerometer to change the modulator level and feedback, and the crank to tweak the carrier feedback.
One more thing I want to add in this part: While hacking on the FM algorithm I ran into a lot of bugs that I could hear but I wanted to be able to see them as well. Usually I'd use an audio routing app so that I could record the output of the Simulator to disk and look at it in an audio editor, but I thought it'd be fun to have the audio waveforms on the display as well (and I was curious how well it would perform). Now instead of using the C FMSynth object as our userdata in setGenerator we have a new wrapper struct that contains the FMSynth, a buffer to store 400 samples, and an offset that tracks where we are in the buffer. The fm control object has a drawPlot function that calls playdate->graphics->fillRect 400 times to draw a 1x1 square at each sample position--not the most efficient way to do it, but it seems to perform okay.
The other way to handle parameter changes is through the synth generator's setParameter callback. When you wire a parameter up through this interface, you can automate it with a signal--an envelope, an LFO, or (new in 1.12) a custom signal. In this section's demo, we've replaced the parameter setters on the fm object with a setParameter handler in the generator callbacks, and attached an envelope to the fm synth's modulation level. And we've implemented a custom signal that follows the crank position and connected that to the synth's pitch modulation input.
Like the custom synth generator, custom signals are callback-driven. You create a custom signal with the function
The only required argument, and the only one we're using in this demo, is the signalStepFunc which returns the signal's value at the end of the next render cycle. The iosamples and ifval arguments are used, if necessary, to provide a value at a sample offset in the middle of the cycle for sample-accurate timing. Here's our very simple step function:
All we do is return a value from 0-1 describing how far the crank is from its 180 degree position at the bottom of the device. Plug that into the pitch modulator and you've got a bitchin' whammy bar.
If you listen closely to the previous demo (and you have better hearing than I do) you may notice a little bit of noise. Here's what it looks like in a recording:
What's happening here is our fm_setParameter() callbacks are called at the beginning of the render cycle, once every 256 samples. In our demo the modulator level is set to the envelope level and holds that value for the entire cycle, then jumps to the next value on the next cycle, causing an abrupt shift in the output. This is familiarly known as "zipper noise", and an easy way to fix this is to interpolate from the old value to the next to avoid the discontinuity.
I'm working on the last installment, fyi. Instead of the DX simulator which does a lot of weird stuff to try and mimic the DX's very fussy envelopes I'm doing a simpler 4-op FM synth driven by our own ADSR envelopes. But I'll post the DX one, too..
I'm also still really interested in how you connected the keyboard to it. I assume it's through the mic input, and the keyboard is sending voltages, like CV?
very cool to see stuff.. I am pretty deep down the synth rabbit hole myself with a bunch of eurorack and hardware synths. Curious also about the teensy hooked up to it, I am guessing it essentially is letting you get midi in and out of the playdate if you want? being able to talk midi to other devices would really open playdate up for some fun synth apps. I think my next playdate project might have to be a little groovebox app.
So I'm opting for more description in the posts here instead.
