Malloc Log Map is Broken in Windows Simulator

[ProfessorSil]

I was recently testing memory constraints of the Playdate to try and configure my own memory management stuff appropriately and I discovered that the Map tab of the Malloc Log window does not behave at all like it does on Mac.

I set up a simple test game where I can press a button to allocate 64kB of memory and clear it to 0xDD so I can test what this tab looks like at various amounts of usage. As far as I can tell, the visualization does not change almost at all, not matter how much memory I am using. This is with "Automatically Refresh" being turned on (and clicking the refresh button occasionally to be sure).

Screenshots

Low Memory Usage (5.67MB):

High Memory Usage (12.67MB):

With Pause Menu Open (Red block has moved):

Also nothing in the simulator seems to complain if I go over 16MB of memory usage. All of my allocations are going through pd->system->realloc so they are getting tracked (hence why the number in the bottom right changes). I have noticed the "Heap Used" number doesn't seem to change much either. Maybe that's related? Or maybe I don't understand what it's supposed to indicate. Also I'm on SDK 2.1.1 (latest)

[jkrenge (Julian)]

Probably not incredibly helpful, but maybe some additional questions can help shed some more light on this. I'm developing on Mac where it works but do have access to a Windows machine, so will attempt to test this.

Questions:

• Have you set your simulator malloc pool to 16 MB?
• Does it capture malloc from device correctly?
• Have you tried turning off auto refresh and only manually refreshing?

And mind sharing your code?

[willco (Will)]

Looking in to this, if I leak in update() the memory map list updates as expected. I'm doing this:

	void *ptr = NULL;
	if ( pd->system->realloc(ptr, 10240) == NULL )
		pd->system->logToConsole("malloc error");

The small/unchanging memory map you're showing is because your heap value is small and that is what is being plotted in this view.

I did confirm the 16 MB limit isn't working correctly on Windows which we'll get fixed up. It should be returning NULL if you hit the limit.

[ProfessorSil]

Hmm, there's definitely something more nuanced going on. Try changing your allocation to something much larger, like 256kB at a time. You'll find that the map tab of the malloc log window does not show the same sort of growth. Also the "Active" value grows much quicker while the "Heap Used" value doesn't seem to change at all. In my case, I have a memory allocator that only allocates 256kb pages at a time and then manages the memory inside those pages internally. So I was never making any small allocations to pd->system->realloc directly

[willco (Will)]

Tested 256, works fine. As long as the heap value is incrementing the map table fills. If you could work up an example that shows exactly what you're doing that would be most helpful. Are you double sure you're not calling malloc directly someplace (instead of pd->system->realloc())? Are you using some library that may be calling it?

[ProfessorSil]

Here's some code you can add to the hello_world example to reproduce the behavior. I said 256kB in my last post, but I suppose I should be clear. I meant 256*1024 bytes at a time, or 262144 bytes. If you do 256*1000 bytes at a time, then the visual works just fine

PDButtons buttons;
pd->system->getButtonState(&buttons, NULL, NULL);
if ((buttons & kButtonB) != 0)
{
	void* ptr = pd->system->realloc(0, 256*1024);
	if (ptr == NULL)
	{
		pd->system->logToConsole("malloc error");
	}
	else
	{
		pd->system->logToConsole("malloc success %p", ptr);
	}
}

Full Source: //// main.c// Extension//// Created by Dave Hayden on 7/30/14.// C - Pastebin.com

[willco (Will)]

Thanks for the code. I did reproduce the issue. The good news is it was already fixed with enabling the 16 MB limiting code. This fix will be in the next release. Thanks for the report!

[ProfessorSil]

I'm curious, was the problem something to do with that specific size of allocation? Or was it allocations over a particular size? The more I looked into this, the more I was surprised it was breaking for only the very specific size of allocation I was doing

[willco (Will)]

The allocation tracker, when run on Windows, has slightly different behavior when an allocation would fail that was then turning around and allocating blocks when it shouldn't have been. There was also a size threshold which would trigger this different path as well. Both of these paths are now (correctly) disabled on Windows.

[dave (Dave Hayden)]

This fix is in today's 2.3 update. Please give it a try and let us know if it doesn't address the problem. Thanks!

[omoikane]

I just tried enabling malloc pool in 2.3.0, and it said "not enough memory" while trying to load my game:

But my game worked in 2.2.0 with malloc pool enabled, and also worked on the actual device (2.2.0 and 2.3.0). Has the memory requirements changed?

[willco (Will)]

Yes, Malloc Pool didn't correctly work before 2.3 on Windows and would allow you to allocate memory beyond the 16 MB limit. I'm not sure why it was working on device, but I would look into this further. Looks like you're right up against the limit.

[omoikane]

I started a new thread for this and added a test case there:

Malloc pool failures with arrays

I have a program that used to work with 2.2.0 simulator with and without malloc pool enabled, and also works on the actual device. With the 2.3.0 simulator, the simulator fails with "not enough memory" on an import line. The file I am trying to import contains many large arrays and apparently the 2.3.0 simulator didn't like it. I have replicated what I am doing in this simplified project: memory_test.zip (1.3 MB) You can see that main.lua tries to import data.lua, which contains a large_arr…

I can't tell if I was close to the limit, way over the limit, or something else.