I cannot get pd.datastore to read large numbers (1.12.0)

SDK Bug Reports
1

I am trying to read layers[1]["data"] which is from tiled (so I don't have control over the data as I want to read it's raw files), and the blasted file format combines 6 datapoints (tileset ID, tile ID, flipX, flipY, flipBoth, rotate) into a single 32bit value whereas any sane developer would have separated them

as you can see in the screenshot, the data loaded does not match the data in the file and seems to be cut off

image
image1131×928 29.1 KB

{ "compressionlevel":-1,
 "height":5,
 "infinite":false,
 "layers":[
        {
         "data":[1, 2, 3, 0, 3, 2, 2147483649,
            4, 5, 6, 6, 6, 2147483653, 2147483652,
            0, 8, 9, 13, 2147483657, 2147483656, 0,
            1073741828, 7, 10, 11, 2147483658, 2147483655, 3221225476,
            1073741825, 1073741826, 1073741827, 12, 1073741827, 1073741826, 3221225473],
         "height":5,
         "id":1,
         "name":"main",
         "opacity":1,
         "properties":[
                {
                 "name":"bgcolor",
                 "type":"int",
                 "value":0
                }, 
                {
                 "name":"z",
                 "type":"int",
                 "value":1
                }],
         "type":"tilelayer",
         "visible":true,
         "width":7,
         "x":0,
         "y":0
        }, 
        {
         "data":[0, 0, 0, 14, 0, 0, 0,
            0, 0, 0, 0, 0, 0, 0,
            0, 0, 0, 0, 0, 0, 0,
            0, 0, 0, 0, 0, 0, 0,
            0, 0, 0, 0, 0, 0, 0],
         "height":5,
         "id":3,
         "name":"elevator",
         "opacity":1,
         "properties":[
                {
                 "name":"z",
                 "type":"float",
                 "value":1.5
                }],
         "type":"tilelayer",
         "visible":true,
         "width":7,
         "x":0,
         "y":0
        }],
 "nextlayerid":4,
 "nextobjectid":1,
 "orientation":"orthogonal",
 "renderorder":"right-down",
 "tiledversion":"1.9.0",
 "tileheight":40,
 "tilesets":[
        {
         "firstgid":1,
         "source":"waitingroom-tiles.json"
        }],
 "tilewidth":40,
 "type":"map",
 "version":"1.9",
 "width":7
}
2 
function bignumber(number)
	return 								string.format("%.f", number)
end

function printBigNumbers(data)
	printTable(data)
	for i = 1, #data do
		print(i .. ": " .. bignumber(data[i]))
	end
end

-- the actual code
	local JSON 							= pd.datastore.read(filename)
	
	printTable(JSON["layers"])
	printBigNumbers(JSON["layers"][1]["data"])
	printBigNumbers(JSON["layers"][2]["data"])
3

DECODING GRID: 0.0 (BEFORE: 2147483648 - 2.147484e+09) AT X: 7 Y: 1 FLIP X: true Y: false D: false R: false BECOMES SET: 0 ID: 0

When I put 2.147484e+09 into decimal I get 2147484000 but bignumber() is getting 2147483648, and the source data is 2147483649

4

It's a little hard to tell what's happening here, but it seems like your integer data is being read from the JSON as float, and then converted to integer with some precision loss. Am I understanding this correctly?

5

Here's what printTable shows immediately after pd.datastore.read

image
image580×787 12.7 KB

so it does look like it's being rounded into a float, thus ruining the data

6

Yeah, unfortunately JSON can be ambiguous about numeric types, but we might not be doing the right thing here. I'll file it and we'll take a look. Sorry for the trouble!

7

Thank you. I'll try to parse this part of the file manually. Is there a way to force a variable to handle a large number? Or will bypassing the JSON library do that automatically

8

I don't think it's the variable; it's the conversion process that's somehow going JSON -> float -> string -> integer. The string representation only retains 6 decimal digits of precision, but you need 10, so your low digits are being truncated and zero-filled. At least that's what I can gather from your screenshots.

I'll let you know as soon as I hear of a fix or workaround.

10

So apparently tiled can export as lua, but I'm still getting the same problem!

every big number starting with 214 will read as 2147483648
every big number starting with 322 will read as 3221225472

return {
  version = "1.9",
  luaversion = "5.1",
  tiledversion = "1.9.0",
  class = "",
  orientation = "orthogonal",
  renderorder = "right-down",
  width = 7,
  height = 5,
  tilewidth = 40,
  tileheight = 40,
  nextlayerid = 4,
  nextobjectid = 1,
  properties = {},
  tilesets = {
    {
      name = "lounge",
      firstgid = 1,
      filename = "waitingroom-tiles.json"
    }
  },
  layers = {
    {
      type = "tilelayer",
      x = 0,
      y = 0,
      width = 7,
      height = 5,
      id = 1,
      name = "main",
      class = "",
      visible = true,
      opacity = 1,
      offsetx = 0,
      offsety = 0,
      parallaxx = 1,
      parallaxy = 1,
      properties = {
        ["bgcolor"] = 0,
        ["z"] = 1
      },
      encoding = "lua",
      data = {
        1, 2, 3, 0, 3, 2, 2147483649,
        4, 5, 6, 6, 6, 2147483653, 2147483652,
        0, 8, 9, 13, 2147483657, 2147483656, 0,
        1073741828, 7, 10, 11, 2147483658, 2147483655, 3221225476,
        1073741825, 1073741826, 1073741827, 12, 1073741827, 1073741826, 3221225473
      }
    },
    {
      type = "tilelayer",
      x = 0,
      y = 0,
      width = 7,
      height = 5,
      id = 3,
      name = "elevator",
      class = "",
      visible = true,
      opacity = 1,
      offsetx = 0,
      offsety = 0,
      parallaxx = 1,
      parallaxy = 1,
      properties = {
        ["z"] = 1.5
      },
      encoding = "lua",
      data = {
        0, 0, 0, 14, 0, 0, 0,
        0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 0, 0, 0
      }
    }
  }
}
11

I think this is a combination of problems.

Our version of Lua uses 32 bits to represent numbers, and 2147483647 is the maximum value of a signed 32-bit integer. Higher numbers get converted to floats, which will be less precise. You just can't represent a number this large in a single Lua variable on our platform, unfortunately. Standard Lua is 64-bit, which must be what Tiled expects.

From what you describe, sometimes your numbers are being truncated to math.maxinteger, and other times they're being converted to floating-point and losing precision. But it's not clear to me why there are two different behaviors, and I don't know what the significance of 3221225472 is.

There are Lua libraries that can help (search for "bignum" or "arbitrary precision") but I don't know how well any of them work on Playdate. If you find a good one, let us know!

12

Checked bignum and it looks no where near compatible + above my skill level to make it compatible.

I was able to parse the JSON manually (luckily Tiled does not minify the JSON) so I can detect when the value is > 2000000000 and fall back to using this data

the problem is it's a string

trying "bint" and "arbitrary precision"

13

Manual parsing is no fun, but I hope you find a solution. Sorry we don't have a better answer at the moment.

14

maybe try Lua tonumber(str) which copes with exponents and other numbers encoded as strings.

> = tonumber("123") + 25
148
> x = tonumber("123.456e5")
> print(x)
12345600
15

BEFORE: 2147483649 AFTER: 2.147484e+09

Yeah there is no way tonumber was going to work given the topic.

16

I only need a single operation, the result of the text value subtracting 2147483648 exactly/only, and the result will always be lower than that so the result can fit inside a normal number

I have no idea how to integrate a C function, but would C be able to parse the number properly/without rounding it thus destroying the value, and return the result of the subtraction?

I tried splitting up the value into each digit and subtracting 1 2147483648 times but that took too long, as I suspected it would.

17

Ah I see! It would help with reading but you've already lost precision whilst writing.

18

I managed to hack together the hackiest large number support by truncating the first digit and handling it separately (1 = flip Y then subtract 73741824, 2= Flip X then subtract 147483648, 3 = flip both then subtract 221225472). This means I have an upper limit of 999999999 tiles before the math breaks down but I'm sure I'll never need that many (may future programmers have mercy on my soul)

image
image2920×1424 210 KB

3 Likes
19

This may or may not be related, but I've had problems writing Lua's math.huge to JSON too. It's a constant representing +infinity, but instead of the JSON serializer interpreting it as the max_int or something, it write as 'null'. So I've switched to using my own arbitrarily large number instead which has more predictable results.

20

I think in lua math.huge isn’t a proper value, It’s only a representation of infinity (a number that is greater the any other), if you print out the value you will see it only prints inf. If you want to get the max integer value you need to use math.maxinteger

1 Like