ESP32: why does malloc can only alloc 126888 bytes with 402540 bytes heap?

Question

I need a buffer of the size 153600 bytes in my ESP32 project, but I could not create a static buffer in code nor can I malloc the buffer.

The ESP32 itself reports:

  ESP.getHeapSize()  // = 402540
  ESP.getFreeHeap()  // = 376980
  ESP.getPsramSize() // = 0
  ESP.getFreePsram() // = 0

malloc(153600) returns NULL; I wrote a little loop which decrements by one and retries to malloc. The first time it works is malloc(126888) so this seems to be the maximum buffer I can get.

(I tried it with different boards; one from eBay and one from a genuine distributor - no differences.)

Clearly the heap size is big enough. So I'm thinking that the RAM is possibly split into different chunks and no one is big enough for my buffer?

Is there any documentation which explains this behavior?

Answer 1

Yes, the ESP32 memory is arranged in a number of different chunks. Those chunks are all aggregated by the memory manager to appear as a single heap, but they're actually separate.

As the ESP-IDF documentation mentions:

At startup, all ESP-IDF apps log a summary of all heap addresses (and sizes) at level Info:

I (252) heap_init: Initializing. RAM available for dynamic allocation:
I (259) heap_init: At 3FFAE6E0 len 00001920 (6 KiB): DRAM
I (265) heap_init: At 3FFB2EC8 len 0002D138 (180 KiB): DRAM
I (272) heap_init: At 3FFE0440 len 00003AE0 (14 KiB): D/IRAM
I (278) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (284) heap_init: At 4008944C len 00016BB4 (90 KiB): IRAM

And of course you have some heap fragmentation from other threads allocating and deallocating memory, so the biggest contiguous block of memory you can allocate will be smaller than the biggest region.

To further illustrate I found this handy memory map on the Arduino forums:

There you can clearly see the SRAM is split into different chunks. The main reason for this is to better allow concurrent access by the two cores in the dual-core variants of the chip.