A list of helpful libraries and code

SDK Development Discussion
72

So when I was working on the LDtk level loader, something that was bothering me was that parsing a level was not fast enough. To solve this issue, after reading a level, I was exporting the result in a lua file that can be used instead of the json file.

I wanted to share a simpler version of this that could be used more generally. But the general idea:

  1. When running in the simulator, after parsing a file we write the result as a lua file in the save folder (In the SDK folder)
  2. The lua file is copied in the project folder
  3. When running on the console if the lua file is present, load it instead of parsing the file
  4. Save the frame!

The first piece of the puzzle is the following function that export a table as a lua file

function writeLua( filepath, table_to_export )
	assert( filepath, "writeLua, filepath required")
	assert( table_to_export, "writeLua, table_to_export required")

	local file, file_error = playdate.file.open( filepath, playdate.file.kFileWrite)
	if not file then
		print("writeLua, Cannot open file ", filepath," (", file_error, ")")
		return
	end

	local _isArray = function( t )
		if type(t[1])=="nil" then return false end

		local pairs_count = 0
		for key in pairs(t) do
			pairs_count = pairs_count + 1
			if type(key)~="number" then
				return false
			end
		end

		return pairs_count==#t
	end

	local _write_entry
	_write_entry = function( entry, name )
		local entry_type = type(entry)

		if entry_type=="table" then
			file:write("{")
			if _isArray( entry ) then
				for key, value in ipairs(entry) do
					_write_entry(value, key)
					file:write(",")
				end
			else
				for key, value in pairs(entry) do
					file:write("[\""..tostring(key).."\"]=")
					_write_entry(value, key)
					file:write(",")
				end
			end
			file:write("}")
		elseif entry_type=="string" then
			file:write("\""..tostring(entry).."\"")
		elseif entry_type=="boolean" or entry_type=="number" then
			file:write(tostring(entry))
		else
			file:write("nil")
		end
	end

	file:write("return ")
	_write_entry( table_to_export )

	file:close()
end

To actually parse or load the lua file directly I have the following code

-- set _enable to false to always load the original file
local _enable = true

-- folder in the project folder where the pre parsed file will be
local _folder = "preParsed/"

function parseFile( parser_fn, filename, ...)
	local pdzFilename = _folder..filename..".pdz"

	if _enable then
		if playdate.file.exists( pdzFilename ) then
			return playdate.file.run( pdzFilename )
		else
			print( "parseFile(): The following file is not pre-parsed", filename)
		end
	end

	return parser_fn( filename, ...)
end

if playdate.isSimulator then
	parseFile = function( parser_fn, filename, ...)
		local result = parser_fn( filename, ...)

		-- save result in lua file
		local luaFilename = _folder..filename..".lua"
		playdate.file.mkdir( luaFilename:match("^(.-)[^/]*$") )
		writeLua( luaFilename, result)

		return result
	end
end

To use it I simply replace a call to a parsing function with it

So for example instead of
level = json.decodeFile( "level_1-1.json" )

I would call it this way
level = parseFile( json.decodeFile, "level_1-1.json" )

Using "level_1-1.json" from the SDK example as a comparaison, loading the pre-parsed lua file is 5 times faster on the playdate than parsing the json file normally. For the svg parser I posted earlier the advantage is even more pronounced since this is 10 times faster.

Big caveats
The biggest drawback of this technique is that you need to be aware of the cached files otherwise you might have edit the original file and the game will still load the previous pre-parsed version. It get even trickier since at the moment files are not deleted when uploading a game to the playdate (so you might delete all the pre-parsed in your project but on playdate it will still load some file you don't even know are still there)

Tips
I created a symbolic link in my save folder to the "preParsed/" folder in my project so that I don't have to manually copy the files.
I also wrote a function in my project to pre-parsed all files in one go. Right now I call it when the game start since there is not that much files but later I might simply call it from the simulator console when I need to create a build.

2 Likes