Lua load function seems to expect a second argument. Why?

SDK Get Help
1

SDK 1.9.1 Mac

Per the docs, the entire Lua 5.4 API should be available.

I'm having trouble calling the load function.

Code sample inspired by this stackoverflow answer:

local func, err = load("return function(a,b) return a+b end")
if func then
  local ok, add = pcall(func)
  if ok then
    print(add(2,3))
  else
    print("Execution error:", add)
  end
else
  print("Compilation error:", err)
end

This should print 5, but this is the Simulator console output:

level.lua:32: bad argument #2 to 'load' (number expected, got no value)
stack traceback:
	[C]: in function 'load'
	level.lua:32: in function 'LoadFile'
	main.lua:6: in function 'startGame'
	[C]: in function 'xpcall'
	main.lua:10: in main chunk
main.lua:15: level.lua:32: bad argument #2 to 'load' (number expected, got no value)
stack traceback:
	[C]: in function 'error'
	main.lua:15: in main chunk
main.lua:15: level.lua:32: bad argument #2 to 'load' (number expected, got no value)
stack traceback:
	[C]: in function 'error'
	main.lua:15: in main chunk
main.lua:15: level.lua:32: bad argument #2 to 'load' (number expected, got no value)
stack traceback:
	[C]: in function 'error'
	main.lua:15: in main chunk

Argument #2 should be optional, and also not be a number but a string (parameter chunkname)
Is this a bug in the SDK?

2

It also seems that io.open(path,"r") from Lua cannot be used. I understand this, given that you don't want to expose the entire filesystem to a game. But I'd say the Inside Playdate docs are misleading because it mentions thatthe entire Lua Api should be available

5

Yes, sorry about that messaging. As far as I know we support the full set of Lua language features, but not every library or API that comes with it. We'll try to clarify this in the docs.

2 Likes
6

That would be nice, thanks.

What can you tell me about the load function specifically?

7

I don't think load() is supported in the SDK because the runtime is using only the VM and not the precompiler which translate Lua into byte code. This is mainly for performance reason.

But you can use playdate.file.load() to load a Lua file that was compiled by pdc
https://sdk.play.date/1.9.1/#f-file.load

Can you explain a bit more the reason why you would like to use load()?

1 Like
8

In fact, I think it could be a good decision to not support load from a security standpoint.

My use case is very specific. I'm porting an older Lua project. Loading a level file that contains parts Lua tables and parts proprietary format. I'm parsing the proprietary format myself and would like to load the Lua bits.

The solution for me is to simply split this to multiple files, and convert the Lua tables to json using json.encode()

A nice-to-have would be a binary format for level data maybe. The json objects are huge for tile based
level data

Out of curiosity I would be interested in a code sample for file.load and how to create those pdz files

9

I would suggest to not use json but instead keep using lua. This would be more faster to load.
For some project where I actually parse files, I sometimes export the result as Lua file that I can load later because it is so much faster to do so (https://devforum.play.date/t/a-list-of-helpful-libraries-and-code/221/72). I would imagine that using Lua file would even be faster than binary file actually (that you would still need to read and convert as Lua table anyway).

If a lua file in your project is not called by any other file using import, pdc will save it as a pdz file (which is mostly Lua byte code) and you can load it using playdate.file.load() or playdate.file.run()

Here an example code where I load a pdz file.
https://devforum.play.date/t/splitting-a-game-into-several-functional-binaries-nics-plugin-manager/1387/5

10

Excellent. The sample was helpful to figure out whether I needed the relative filepath with extension. Thanks!

In the link you posted, you already mention a function that can save a lua table to a file. If you want to go from some static jsons to lua, this online tool might help: JSON Lua table converter - MAGEDDO

11

I realize that this is an old post, but if it helps someone else out, I figured out that the second parameter is the length of the string that you're passing in. I'm starting with simple Javascript code and need to transform it to Lua and execute it. So I'm starting with something like this:
"function nextCard() {return "322";}"

and I have to transform the string to Lua:

"return function() return true end"

And then use this to execute it:

function Evergreen:dynamicExecuteScript(script)
   local length = #script
   local func, err = load(script, length)
   if func ~= nil then
	  local ok, myFunc = pcall(func)
	  if ok then
		 local result = myFunc()
		 return result
	  end
   end

   return nil
end
12

Oops, I didn't realize that worked in the simulator. We include the compiler part of the Lua runtime in the simulator so that you can enter and run code from the console, but I should have disabled it in the load command because that's not available on the device. We don't have a lot of room for the firmware, so we need to trim wherever we can.

Just a heads up, this will stop working in a future update!

13

I was afraid you might say something like that if I mentioned it, but I'd prefer to know sooner than later. If you ever make a PD2, this would be very high on my wishlist of api's to include. Or a simple javascript interpreter. Either one. Thanks.

1 Like
14

Hello! I'm digging this thread back up because I also ran into a similar situation where "load" is expecting a second argument.

While, the "fix" of adding the string length seems to work, it seems that the "hack" is still possible to achieve in the simulator.

Is it still planned to remove that behaviour from the simulator? And if so, is there any way to execute code dynamically generated?

(I am trying to implement Narrator, a Ink lib for Lua on the playdate, and a lot of the conditions and execution logic depends on this function :frowning:

15

Oops, sounds like I never got around to disabling load() in the simulator. One of these days I'll have a chance to clear out all those old bugs. I've only got, let's see.. 257 assigned to me, shouldn't take too long. :stuck_out_tongue_winking_eye:

No, I don't expect we'd ever include that as part of the core functionality, but I think it would be possible to put the Lua compiler in a C API project and implement dynamically generated code that way. When you add a function to the Lua runtime from the C API you use a lua_CFunction which takes the lua_State as an argument. Once you have that you should be able to call luaL_loadstring() on your Lua source and get the compiled code on the stack in the Playdate's Lua runtime. Something like (completely untested code here)

int compileCode(lua_State* L)
{
    const char* code = pd->lua->getArgString(1);
    return luaL_loadstring(L, code) == LUA_OK ? 1 : 0;
}

add it with

    pd->lua->addFunction(compileCode, "compile", NULL); // XXX - add error checking!

then call it like

    local func = compile("print('it worked! I can\'t believe it!')");
    func()
1 Like
16

I completely understand the struggle of going through a never ending task list... I hope my message was not taken the wrong way, I was just wondering if this changed, or if I missed an update going through changelogs.

As for the C example, thanks! I have never dealt with C before, but I guess this is a good starting point for me to use your example and try it out!

17

I got up this morning and thought, "I did half the work here, might as well finish the job." Wasn't quite as simple as I'd thought but it was an interesting journey. First step is setting up the project then getting all the Lua code we'll need. This is pretty simple, you just try to compile then see which functions it complains are missing, find those functions in the lua source, add those files, repeat. In the end it was only complaining that an abort() call in there was linked to low-level system stuff that we don't have on playdate, so I added our own abort() implementation that calls pd->system->error(), and it compiled!

Running that in the simulator I next ran into a parse error in the luaL_loadstring() call. Classic string escaping goof: pdc unescapes that backslash in can\'t, so when it goes into the compile() call it looks like

  compile("print('it worked! I can't believe it!");

which throws a syntax error. I brute forced it by changing that to can\\\'t in main.lua. If you're getting strings from a text file or such you probably won't have this problem, but it's something to be aware of.

Next problem: it compiles but the generated bytecode doesn't run, looks like it's corrupted. Eventually I remembered that due to a historical accident our opcodes are shifted from standard Lua, so I copied over our lopcodes.[ch] files and got past that hurdle.

Now the code runs in the simulator! ..but not on the device. I connected the hardware debugger and got to the crash location, things look a little weird.. The Lua global struct doesn't look the same there as it does in the main firmware. The place it diverges is TValue l_registry, TValue is a union of types including numbers, and we're using 32 bit numbers instead of 64 on Playdate. Bingo. Setting LUA_32BITS to 1 in luaconf.h fixes that.

So here you go, a replacement for load():
evalC.zip (215.2 KB)

Disclaimer: This code is completely unsupported, may break at any time in the future. For entertainment purposes only. Past performance does not guarantee future returns.

4 Likes
18

This is, once again, way more than anyone could expect :star_struck:. Thank you so much for the very detailed exploration (and sorry for whatever task you had to skip to fulfill that brain worm of a problem :grin:).

I'm completely new to C, so unfortunately I'm running into problems compiling/building it (I get an error C2040 about a "lua_State differs in levels of indirection from 'void *".

I'm guessing I'm messing up my CMake setup and my whole solution build since the examples in the SDK all work fine, so I'll keep looking around to get it to run on my end :smiling_face:

19

oh, right! That was one other thing I had to do, change

typedef void* lua_State;

to

typedef struct lua_State lua_State;

in C_API/pd_api/pd_api_lua.h in the SDK folder. We'll get that fixed in an upcoming SDK update, assuming it doesn't cause any problems. I think anonymous structs are fine in C if you're only dealing with pointers to them.

2 Likes
20

LEGENDARY!

One last issue I ran into is that is complained about abort being already defined. So I was trying to look when but apart in the main.c, couldn't find out. Commenting this block made it compile and it runs in the Simulator, but crashes on the console... Getting close, but still struggling on my end :sweat_smile:

21

@dave thanks for sharing this.

I discovered this thread because I found an elegant solution to a problem I was working on that involves string.dump which works in the simulator but not on device.

It would be great to have dump/load available somehow it a more reliable way.

I understand that this probably isn't going to make it into the SDK, but what about making this modified Lua compiler available publicly?

EDIT: to clarify, I meant in a GitHub repository under the Panic org that is kept maintained as it develops internally and evolves with SDK versions.

22

This doesn't seem to have been updated yet. I checked 2.1.1