SDK 2.0 b2 - PDC produces pdx with broken binary

PDC produces pdx with broken binary for any example in C_API/Examples/. It can not be run on my hardware - getting error "CAPI Handler function not found..." for each example.

PDC in SDK 1.13.7 work OK, produces normal binaries that works fine on the device.

My hardware:

build=9da6486e9095-1.13.7-release.153702-buildbot-20230504_180456
boot_build=9da6486e9095-1.13.7-release.153702-buildbot
SDK=1.13.7
pdxversion=11300
serial#=PDU1-Y000235
cc=9.2.1 20191025 (release) [ARM/arm-9-branch revision 277599]

Host: MacBook Pro with M1, macOs 13.3.1 (a).

Video screencast with reproduction of the problem (watch in 4K because of text).

From the OS 2.0 guide:

Playdates on OS 1.x will not be able to launch your 2.0 build

We know OS 1.x won't launch 2.0 builds, but it should say "Updated Needed" (indicating a correct 2.0 build that 1.x knows not to launch) rather than "CAPI Handler function not found" which indicates an incorrect build.

1 Like

What are you using to build the C part of your PDX?

As you can see in the video, just make in the directory with example.
Arm toolchain installed with playdate sdk and pdc.

Maybe it because my device is early dev revision? Check out the serial number. Could it be?
I’m sure not, because the same layout but who knows…

Hm, I'm not having any luck reproducing that here. Can you check the pdxversion field in the pdxinfo file inside the .pdx game folder. (Whew, that's a lot of pdxes..) It should be 20000 indicating that it was compiled by a 2.0 SDK. That tells the launcher that it can't run the game on a 1.x OS. If the game was somehow build and packaged by (say) the 1.13.x version of pdc, then that field will be 11300 and the launch will think it can run it even though the pdex.bin file is incompatible.

Anyway, how I can test validity of my elf to ensure that it will loaded normally with new loader with relocations?
Also, what relocation model is preferred? PIC or ROPI-RWPI? (llvm reloc-models for example)

Dave, might I ask you to validate my elfs and tell me what's wrong with it, please?

Brothers Elfs.zip (31.5 KB)

There are two relocation models - pic and ropi-rwpi.
And two various linkers used.

I just want to prepare my system for 2.0 with new loader.

Thank you in advance, I appreciate the time you will take to consider this matter.

Can you check the pdx folders and verify that there's a pdex.bin file in there and that it's larger than 0 bytes? There's a missing dependency in the C_API/buildsupport/common.mk file in the SDK which causes it to miss that if you have parallel builds enabled. (see Missing/incorrect dependencies for simulator DYLIB rule in common.mk - #6 by DidierMalenfant) Here's the updated version:
common.mk.zip (1.8 KB)

You shouldn't need to change compilation options, no need for -fPIC or such. We decided not to use the elf relocation format after all, didn't want to force you to include symbolic information in your game that you might not want to have public. pdc adds a relocation table in a custom format, and the loader is much simpler.

2 Likes

If pdc is now using a custom relocation format, does that mean there's no longer going to be a pdex.elf file, per this page? So the necessary recompilation will update the pdxversion and a relocation table inside the pdc-created binary, but no other changes? That seems simpler, and nice. I'm curious because we're having trouble getting 2.0 to work with Rust, and it seems like we shouldn't need any changes on our side if that's true; @fzzr is our expert, but has been stuck on this issue.

1 Like

Oh! I didn't realize the build system had changed, but of course it did--we have to use the relocation table from the elf file to generate our own stripped down version. (My first hacky version of this just compiled it twice with two different target addresses then diffed them to see where it needed to fix up. Using the elf is much cleaner. :slight_smile:) So that page is correct. On 1.0 our build script compiled an elf file then generated Source/pdex.bin via objcopy, and pdc would copy that to the output folder. On 2.0 pdc does the elf->bin conversion, so the elf file has to be copied to the Source folder first.

1 Like

@dave, do you think similar binary format changes are likely in future Playdate updates? Some people are still working on figuring out the 2.0 format for Rust, but no luck so far. I don't want to rewrite my game in Lua, but I don't want to risk my game being obsoleted, either.

I hope we won't need to make any more changes, and if we do we'll keep support for older builds as long as we can. What happened with 2.0 is the new hardware maps external memory to a different address, so we had to add a relocation table to the pdex.bin file. If you have a v1 device you can still run games with C API components compiled with the old SDK, but there's no way for the v2 hardware to do that. I can't predict the future but I also can't think of what would require another change to the binary format.

Oh, and I see I said

You shouldn't need to change compilation options…

up there. As we discovered with "error: compress failed: buffer is empty" error from pdc in SDK 2.0 beta 2 - #22 by DanB91 it turns out you do need to add the -mword-relocations flag to your compile options (and it's now in the SDK's C_API/buildsupport/common.mk file) so that it doesn't generate any "split" addresses that our loader can't deal with. Support for that type of address load is something we could add in the future.. but I don't think it'd be worth the hassle.

1 Like

Seems to I need help.
I tried really many options and didn't understand where I was wrong. So I have no idea for now.

My elf before pdc have correct entry-point with correct (as I suppose) relocation section.
After pdc a new elf foo.pdx/pdex.bin does not contain anything. Not really anything, mostly seems to problem with relocations.
But no errors by pdc.

There is dummy isolated example (/playground).

1 Like

There's a bug where the compiler can't handle pdex.elf files with more than one segment, and it looks like that's the problem here. I'm not sure why there's a second segment in the file, doesn't look like there's anything in there, but its existence causes the compiler to forget how large the binary is. I posted a fixed macOS build of pdc here: "error: compress failed: buffer is empty" error from pdc in SDK 2.0 beta 2 - #17 by dave Can you put that in your SDK's bin folder and give it a try? As far as I can tell, loading and running works correctly with that in place. You get an error after that because an update callback hasn't been set and there's no Lua main.pdz file, but that's a separate issue. :slight_smile:

1 Like

Dave, already tried yesterday. Same result.
I’ll try tomorrow with less dummy “project” with update-cb.
And I don’t get a speaking error, just crash and “need reboot, press A”.

Double-checking that, I first ran into the problem that gatekeeper isn't letting me run that pdc build from the SDK/bin folder (but it does let me run it from the Downloads folder??) so I had to sign it with codesign -f -s - pdc. But after that I still don't get the same pdex.bin I did before. It looks like that pdc build didn't actually have the bug fix. :frowning:

Let's try this again: pdc.zip (793.3 KB)

And here's the resulting pdx I get
foo.pdx.zip (1007 Bytes)

2 Likes

Seems to IT WORKS. I’ll test and tell you later today.
Also instead of codesign I’m using just xattr -d com.apple.quarantine.

2 Likes

@Dave, I’ve tested and have to say ^that patched pdc works properly.

I have no classic giant bunch of tests, just

  • not inlined functions,
  • heap allocs, pointers to various parts of prog
  • two or more sections.

Looks good for me. Thanks, Dave!

Also have to say, I’ve used PIC relocation model.

1 Like