Bridging 2D array between Lua and C

SDK Get Help
, ,
1

I have a game that operates on large 2d arrays for some procedural generation. Most of the game's logic is written in Lua, which I've optimized a lot, but there are some bottlenecks I might move into C.

As far as I can tell, the best way to bridge between the two is going to be registering a Lua class with pd->lua->registerClass that defines a way to set/get data from a 2d array, something like this:

typedef {
  int major;
  int minor;
  short **grid;
} Grid;

static int new_grid(lua_State *L) {
  int major = pd->lua->getArgInt(1);
  int minor = pd->lua->getArgInt(2);

  Grid *grid = pd->system->realloc(NULL, sizeof(Grid));
  grid->major = major;
  grid->minor = minor;

  short i;
  short **array = pd->system->realloc(NULL, major * sizeof(short *));
  array[0] = pd->system->realloc(NULL, major * minor * sizeof(short));
  for (i = 1; i < major; i++) {
    array[i] = array[0] + i * minor;
  }
  grid->data = array;
  pd->lua->pushObject(grid, "c_Grid", 0);

  return 1;
}

// garbage collect, :set() and :get() methods, etc.

static int operateOnCGrid(lua_State *L) {
  Grid *grid = pd->lua->getArgObject(1, "c_Grid", NULL);
  // do something with the Grid
}

pd->lua->registerClass("c_Grid", GridLib, NULL, 0, NULL);
pd->lua->registerFunction(operateOnCGrid, "operateOnCGrid", NULL);

Then, to prepare a 2d table in Lua for operations in C, I'd need to make an instance of the class to pass into an exposed function:

local cgrid = c_Grid.new(height, width)
for row = 1, height do
  for col = 1, width do
    cgrid:set(row-1, col-1, someValue)
  end
end
operateOnCGrid(cgrid)

I wanted to confirm my understanding is correct, and that there's no more performant way to approach this. Right now I'm working with a 40x40 grid, and it's very slow to perform 1600 copies every time I pass the grid from C to Lua or vise versa.

I can avoid the repeated cost of performing this copy in/out by using a c_Grid rather than a default table in my Lua code, but then I incur the overhead of C FFI every time I perform an array index/etc, which could be pretty often. From a quick profile, setting a value in a 2d Lua table is about 10x faster than invoking a method on a class registered from C to do the same thing.

It seems like the answer might just be profiling what the breaking points are for each approach and figure out the most performant way to combine them.

2

I haven't done Lua code in a long while, but I think you want something like this:

  1. have lua allocate a block of memory of type "full user data" for you. This is your array.
  2. create your own indexing lua function in C.
  3. set that function as a variable named __index in a table
  4. attach the table to the user data as a metatable

PD's registerClass is doing something like this (and much more) as a convenience. You might be able hijack it to do what you want.

Also, consider making your struct like this because mallocs of mallocs is inefficient.

typedef {
  int major;
  int minor;
  short grid[];
} Grid;

Grid *grid = pd->system->realloc(NULL, 2*sizeof(int) + (major*minor)*sizeof(short));

index with grid[y*major+x]

3

Thank you for the advice, I appreciate you replying!
Is there any benefit to defining the indexing function as __index in the metatable, rather than as a get (or whatever) method like I am now? Unless I switch this to a 1-dimensional array that I index using row * minor + col, like you mentioned, I am not sure I'd be able to implement the 2-dimensional indexing with __index. I'm not too familiar with Lua internals, so I'm sorry if I'm misunderstanding how this works.
Offhand, would you expect your approach to be more performant than registerClass?

4

Doing the 2D to 1D calculation in lua will require even more lookups to get the size variables so I’d say that option is right out.

You can do 2d indexing with __index if your indexing function takes a two element array as the key. Lua tables have a fast path for integer indexes so it won’t be too inefficient.

If you want to do it the Get way… making a function call on user data still requires __index to be implemented, just differently. The runtime needs it to lookup the name “Get” to get a pointer to your get function. Implementing it this way requires __index to be a table with the “Get” pointer variable in it.

Sorry for the vagueness. I’m super rusty. I haven’t touched lua since 5.2.

5

No problem, you've explained it very well!

I'll take a look at implementing __index using a two-element table, still using PD->registerClass and see how that changes performance.

6

I was describing a bottom-up design.

If you’re leveraging PD’s register class, __index may already be defined for you. It’s part of how “Programming in Lua” suggests you make class hierarchies.

You may have to make your userdata a member variable of the class and make your accessor methods the PD way.

Ultimately, if you really need performance you should be doing it mostly in C plus a lua interface. I don’t believe that PD is using luaJIT so performance will always be worse.