tl;dr
...I kid! This looks great and your detailed breakdown is very interesting for both design and technical implementation!
3 Likes
How wonderful! This sort of totally wild idea really floats my boat.
I'll follow your progress with great anticipation.
Well done so far. I really appreciate the deep insight into your process.
1 Like
Glad the idea seems to grab people!
The Playdate is so cute, whimsical and cheerful. I really want to make a game that saps all that joy out of it.
My original mobile (possibly ARG?) game concept—long before Playdate—had NO GRAPHICS. Just VFD-looking text (green with occasional red "jump scares") and audio inspired by shortwave "numbers stations." (Look them up and listen if you want to be creeped out.)
When the Playdate was announced, the crank immediately made me think of this never-finished horror radio-tuning concept.
But the PD screen was so cool! I imagined that the silvery LCD would give photos the look of creepy old daguerreotypes, and thought about marrying the signal-hunting to a big scrolling image. So here we are: the image component has grown to BE the main game, and the original all-text interface lives on in the bottom UI bar.
Long before the SDK was out, I made a JavaScript/Canvas-based proof-of-concept for the HDR shadow/highlight tuning, requiring a fast desktop processor. I simulated the crank by dragging the purple bar back and forth:
When I eventually got a Playdate, the concept smacked hard into the wall of what that little CPU could actually do. Not to mention, the lack of ability to even load a grayscale image internally.
But I didn't give up. I spent several days—in an actual cabin in the woods with no neighbors and no Internet—trying to optimize into something playable. Success! (Followed by months of on-and-off reworking to make it look and work better. The last 5% of the experience was probably 95% of the work. So much for 80/20!)
I haven't decided how to handle the audio yet—actual voices, I hope? I intend to compress them to super low distorted bitrates, because that in itself is creepy! And gentle on the RAM.
(And true to my Playdate roots... there is already a clock in the game. Just an Easter egg some players may discover.)
3 Likes
This is quite something! Can I ask you to elaborate on
I'm not sure I understand what you mean here.
2 Likes
I read it as three pieces? But yeah I had to think hard about that paragraph.
1 Like
Ah, the "bits" are the three 1-bit base images, yes - that must be it! The double use of bit threw me a bit 
2 Likes
Right, three 1-bit "source pixels" go into the recipe to become an 8x8 dithered zone on-screen.
The big Photoshop document has 256 gray levels, which then get indexed (using a custom CLUT and Diffision dither turned up to 100) into an 8-color (3-bit) intermediate step. But even then it looks like more than 8 grays because of the dithering, along with all the texturing contained in the artwork.
Then I extract each bit as a separate PNG for the SDK to import. Re-assembling the bits uses the various "recipes." (I first sought a generalized solution of a single process, and failed, but that would have run slower anyway.)
The reason there are two sets of three 1-bit images is because the source data exists at two scales, for the sake of the clearest detail when zoomed out. The second set is used only for "1x" and ".5x" (maybe "2x" in future). I'm not yet certain the extra set is worth it, but it can be removed in later testing. I'm autogenerating it for now, knowing I can play with sharpness etc. later on.
Currently, the source pixels are used at their unscaled native size when viewing "1x" and "8x." But even when doubled you can't see obvious pixels because the grayscale dither detail makes them all but invisible:
That's all raw, unblurred, 2x2 source data pixels... but you can't see them!
As for the dynamic range, "full range" from black to white is considered to be a 4-gray sequence out of the 8. Of course, black-to-white is not how it looks in Photoshop: everything is compressed and looks washed out—as seen below. The only reason for something to look full-contrast in the Photoshop working file is if something is in deep shadow and has a very bright light or flame on it. Same reason we need HDR in photography!
(BTW three elements went into this view: first, my modification of a 1665 ant print by Robert Hooke, which I used in some fictional currency I made, and have re-used as a test image ever since. Second, the eye from the pyramid on the US $1 bill—yes, that's legal! And lastly, some crazy nasty fungus I photographed in the woods.)
8 source gray levels facilitates having 5 meaningfully-different dynamic ranges ("noumenon phases" V through IX in the UI) of 4 grays each:
The lowest phase and highest phase (V and IX) are mutually exclusive: when one is fully "tuned," the other is invisible.
Each phase currently equates to one revolution of the crank—but I'm still playing with what feels good.
I also generate two additional "phases" at either end: "too dark" and "too light," so you never feel the crank is artificially limited. Beyond those, you can keep cranking into pure black or pure white forever. Beyond "phase I" and "phase XIII" (three full revolutions of nothingness) an error is shown to discourage you from cranking on... but you still can.
(When playing, you don't need to crank the image to perfect contrast—nor zoom/center on it perfectly. It's meant to feel vague like an analog process, and that makes it forgiving.)
1 Like
I'm still a little unclear.
Can you change the wording to not use the word "bit"? I think that would help immensely.
Sometimes I'm wondering, does he mean 1-bit pixel? Does he mean pieces? Something else?
1 Like
Good idea—I changed it to "binary digits". (That's always what I mean by bit, not "pieces.")
I didn't want to call them "pixels" because you never see them—and each "pixel" you DO see in the game comes from multiple binary bits of data.
But you could call them invisible "source pixels." Six 1-bit images of them, three of which are used at a time.
Some of the "glitch effects" are random flashes of the least-significant source bit at native scale. You can make out flashes of half-seen images in the glitches, taken from anywhere in the panorama. (That bit is the topmost of the 3 source data images framed in blue above. You can vaguely make out the rune and the eyeball, but not too obviously.)
Here's a "glitch" frame showing the ant. My hope is that people will just think it's noise... except once in a while wonder, did they see a flash of... something?
1 Like
Got it! Thanks again
1 Like
Here's a look at the launch card, some rough menu progress, and on-device performance recorded with Mirror.
I'll make a more complete animated "boot sequence" later. I'm trying for a mostly-text, "scientific device" UI feel, without much imagery beyond the pano itself.
I've abandoned using the (A)+(B) combo to bring up the Signal Log. Not discoverable enough. So I added "LOG" to the left of ".5x" in the magnification bar, which you traverse with (B) and (A).
And you can always long-press (B) as a shortcut to get the Log without having to change your zoom.
In this video, I mash (A) and (B) as fast as I can to show zoom performance. And I hold the d-pad down to show auto-scroll speed. (The HDR fading is done via crank.)
As you can see, zooming displays a rough/quick preview immediately on buttonDown, followed just a moment later by the full-quality render. In practice, only the very highest level of zoom-in ("64x") feels slightly delayed (and you won't need to use that much anyway).
And that delay is only the FIRST TIME you zoom in—then it gets faster. Once you've seen a zoom level, it's cached: now you get instant response at all zoom levels (as the video shows), with no need for the rough preview frame. Until you pan, at which time the current zoom is the only one still cached.
Cranking is ALWAYS totally smooth (actually slowed a bit here by Mirror).
Near the end of the video, you can see the full panoramic strip wrapping around, zoomed all the way out.
In the Signal Log, the numbers in the boxes represent how many items are left to find in that mission. With 1 signal sometimes leading to more than 1 new mission, I'm intending the number of "things to find" available at once to start small, balloon in the middle of the game, and then contract back to 1 goal at the end. Some signals will have a bit of story/lore, but I expect some will just be another mission in the chain. And some signals will have to be ONLY story, or the number of available missions will never contract. Those signals will start out with a "0" as soon as you acquire them.
I'm trying to build all that in a flexible way so that late changes in the feel/progression of missions will remain possible after playtesting.
3 Likes
This is a more-final "Signal Log" UI (with early/placeholder content). Along with a demo of new "glitch" effects using the SDK's vcrPauseFilterImage. The existing glitches still happen as always, but this is a new type in the mix occasionally.
Every item listed in the Signal Log is the result of a signal you found. And gives you a new "mission" to find another item.
-
"?" for a newly-discovered "mission"—something new to look for.
-
A number for a mission asking you to find MULTIPLE things. (So "?" means "1" basically—that's most common.)
-
Crosshair icon for things you have successfully found (but you can still go back to review those signals at will, and return to the pano to view the associated imagery).
-
I'll probably use a little dot or grayed-out crosshair for items that don't lead to any new mission (signals with nothing new to find, just story/atmosphere).
The (A) indicator starts on the currently-active "mission." You can scroll it up and down to select a new one and exit the log. Or (B) reverts instantly back to the current mission and exits the log.
Each item you select as you scroll plays its looping audio, along with a live ticker of its text at the bottom. (That's the sound and text that played on the main pano screen when you first found that item.) The final words of the signal are used as the "mission name" in the list.
2 Likes
Progress update: I'm early in the writing phase now, so may not have much to show for a while! Have to resist diving into detailed art before planning. Release date? Who knows! (Halloween would obviously be nice, but it's done when it's done...)
Meanwhile, though, there might be a little bit of tangential OUTSIDE PARTIES lore in Ledbetter's Art&.. More on Catalog. The Roccati Institute appears to have made a mysterious donation from a collection long kept secret from the public...
Which also gave me the chance to make something in Pulp finally. Fun!
4 Likes
I’ll be posting some footage with audio soon. It’s coming along nicely, really feeling like the image, the sounds, and the text ticker are all ONE thing. I hope!
For now, I’ll share how I am generating procedural music...
Interval Signals
The game is partly inspired by numbers stations so I felt I HAD to build my own actual numbers stations into it. Complete with voices speaking code... and—very occasionally—”interval signals”: vaguely spooky bits of music that play between transmissions.
You won’t hear them often, and you will NEVER hear the same tune twice.
Real interval signals are often bits of folk music of various nationalities. So I programmed a system to write widely-varied snippets of folk music.
Most of the randomizations are weighted: I always like my randomness to feel “uneven,” with some events being much more rare than others.
Here’s how it works:
• Random choice of 3|4 or 4|4 time.
• Random span of 1 (usually) or 2 octaves to draw notes from. Never super high or low.
• Random selection of four scales I put in a table: Dorian C, Mixolydian C, Aeolian C, and Aeolian Dominant C.
• Random tempo.
• Random overall volume.
• Random choice of 6 sample-based instruments: Organ, Flute, Chorus, Bells, Out-of-Tune Piano, and the creepiest of all instruments: the dreaded Toy Piano.
• Random choice of 5 ADSR schemes I made (“Soft,” “Staccato,” Woodwind,” “Woodwind/long release,” “Full + fade-off”) that give those instruments different characters.
• Random bitcrusher amount with random undersampling, for a nice radio distortion that varies. At the extremes, the instruments can sound quite strange.
With those factors decided, it’s time for the Playdate to get to composing....
Procedural Music
Each snippet of music is 8 measures long. Usually it loops twice, since repetition adds structure and feels more “musical.” But when the random tempo is slow, it only plays once. I didn’t want the tunes running long.
• Measures are generated in pairs: a random rhythm is built from half-notes, quarter-notes, and rests, then used for two consecutive measures (of 3 or 4 beats each).
• The process has a little guidance against certain unwanted weird outcomes. For instance, the music never starts with a rest.
• The rhythm contains no pitches, but it has both durations and random velocities (note volumes). The velocity variation helps the music feel more “alive.”
• Random pitches from the chosen scale are assigned to all the resulting notes. Each note in the rhythm table has a 50% chance of being the same for both measures. That means the second measure of the pair feels like a variation on the first.
• The final measure is different: it copies nothing from the preceding measure, but is always a single note of random length, always at least a half note, and it can be longer than the other notes ever get—up to the full measure. This long note (or if short, followed by a long rest) is like a period on the end of the musical “sentence.”
• To time the sequence I use this formula: loops * musicSequence:getLength() / tempo. But if my piece ends with rests (which have no meaning in a sound.track, they’re just unused time) the timing will be wrong. So in the final measure, I use note(s) with velocity 0 to fill the time of any final rest.
The result often sounds, to my untrained ear, like real music! And when it doesn’t quite, it’s still a fine “musical signal.”
I then use a low-frequency oscillator modified to ramp the volume gently up from zero and back down again in a sine curve, like a distant signal coming and going. (Most of the sounds in the game do that to some degree.) Occasionally the two-loop tunes will forgo that and play full-volume start-to-finish. But the one-time tunes (less common) never do: they already feel more like pieces taken from the middle of something, not as much like music that stands alone.
The “data stream” (text ticker) at the bottom of the screen always reflects whatever the audio is doing (which makes an audio-heavy game playable with the sound off). In this case, the text flickers a mix of “INTERVAL SIGNAL” and question marks. (As well as the ever-present occasional flashes of brief creepy messages from Out There.)
Stay tuned for some recordings!
5 Likes
Recording time! (Forgive the slightly slow performance: these were recorded from actual hardware using Mirror, which has a little overhead.)
This is the game's built in "Void Monitor" mode which simply plays the game's sounds while auto-scrolling slowly and changing the contrast randomly. It sounds the same as user-controlled gameplay… and of course, I put a clock in the corner, for maximum utility!
A soundtrack to study, relax, meditate, sleep or party to!
And another recording, this time with more music, procedurally generated as explained above:
2 Likes
Audio Layers
The Playdate (“K5 Panopticon Receiver” in the game) is not meant to be a literal shortwave radio, but something mysterious that’s akin to that.
There are 10 layers of sound, of which 2 to 4 are playing at once:
• UI clicks, different for different actions.
• Static sounds accompanying the three kinds of brief visual glitches. Randomized pitch, and louder for bigger glitches.
• Full stereo background loop, > 1 minute, while viewing the pano. (But any constant sound risks being tiring, so the often-visited Signal Log has no background audio.) This is a mix of a couple modified shortwave radio sounds, varying across the image via Perlin algorithm. It also pans randomly in stereo and changes pitch over time.
• Second stereo loop plus random pan, similar to but simpler than the first, running at a different rate so the interaction of the two provides a non-repeating background. This one rises and falls in pitch with the crank, making it really feel like the crank is “doing something”! And I’m using pitch-compensated volume, volume/rate, so it doesn’t get loud and shrill as it rises.
• Signals: the spoken story/mission snippets that come into clarity (diminishing bitcrusher, diminishing volume dropouts) as you zero in on each target object in the image.
• False signals: heavily-modified random snippets from the full library of spoken signals. Played backwards, randomly re-pitched, and bitcrushed. They fade in and out smoothly, and are only sub-segments of the longer signals, starting at any random point—even near the end, causing the false signal to loop and finish with the beginning. Each signal is really a PAIR of WAVs: a “story” snippet, usually followed by a “mission” snippet leading you to look for something new in the image. (This leaves me great flexibility to alter the game flow at any time, changing the order of the missions while leaving the story alone, and vice versa.) False signals are randomly based on just ONE of these two halves.
• Random white noise overlay that comes and goes occasionally.
• An ever-growing library of creepy samples that fade in and out, with various random distortions applied. Additional radio sounds, screams, machinery, footsteps, you name it! I like “uneven” randomness, and these are one example of that: some samples are more rare than others. I want the player to regularly hear/see something new. Or better yet… half-hear or half-see and wonder, What was that?
• Numbers stations. They partly inspired the game, so I couldn’t leave them out! A large range of spoken numeric codes—but only a few different voice processings, to make it feel like you keep hearing the same couple of stations return.
• Interval signal music. Randomized in a guided way to sound like actual music. (See post above, and listen to some real-world interval signal music examples here.)
Audio Variation Axes
For variety the sounds vary in many ways, each one customized to what sounds good for the particular sound:
• Pitch/rate
• Volume (and sometimes rising and falling volume changes)
• Stereo pan (and sometimes pan movement)
• Bitcrush effect (reducing sample rate, depth, or both)
All sounds vary in response to these 5 factors:
• Randomly on their own (but not so strongly that it swamps the user-controlled factors).
• With proximity to a “real” signal (something to find in the image).
• Spatially when panning the image along X and Y with the dpad. This variance is done using a 2D Perlin algorithm.
• With “phase” (dynamic range) as you turn the crank.
• With magnification as you zoom with (A) and (B).
I wanted EACH of those factors to feel responsive, the changes not getting lost in the other 4. I think I’ve achieved that.
All of those factors are also visible in the pano image itself, AND are reflected in the text data stream at the bottom.
Everything you do feels like you are affecting the image AND the audio AND the data stream… because you are. All together as a single process. You’re doing a bunch of stuff, but I think I’ve made it FEEL like you are doing ONE thing: finding things and zeroing in on them.
(Soon I’ll post a live camera demo of the process of zeroing in on a signal, here or on Mastodon.)
EDIT: Oops! 10 layers, not 11... I decided against including the Morse code layer.
3 Likes
Here's an early peak at the likely style of the "stuff" you're searching for. It's all one big, complex scene—a single "place" seen in 360° panorama—and here's a little zoomed-in portion of it:
That incorporates some of my sculpture work:
5 Likes
This is incredibly cool. Love the spooky mysterious vibe. Can't wait to see more!
2 Likes
Any updates on this project? I just saw it during the last Tiny Yellow Machine direct and it looks so cool!
Thanks for all your hard work and sharing so much already
1 Like
Thanks! No major new specifics yet—it’s coming along but my own projects have to come second to client work 
Outside Parties is tied for #1 priority among my own projects though! And although the creative work has just begun, the technical work is mostly complete.
.
And here’s that Community Direct trailer (also winner of Best Trailer in the 2023 Community Awards)—timestamped at 1:28:
2 Likes