Memory leakage caused by String Concatenation

Question

In this link, section "Memory, memory, memory" it's been told that doing many String concatenations in sequence, e.g.: using operator+() multiple times, will cause memory fragmentation, which may lead to the fact that total heap not being totally used.

An alternative I'm suggesting is using strcat() with char buffers and not using Strings at all, and then convert from resultant char buffer to String if a String is needed at the end? Would that be an escape from fragmentation of the heap?

Clarification: I will specify the upper bound of the char buffer beforehand, as if I know the maximum needed/to-be-used size.

Answer 1

If you size the buffer to be able to hold your final string, then yes, that method will avoid the fragmentation caused by the String concatenation function.

Note that String holds the string as a C-style char array internally; it is the concatenation procedure which dynamically allocates and deallocates memory, causing the fragmentation.

Answer 2

Your method would reduce the fragmentation. But will your application need do this once? A few times? Must it run forever? If your application uses malloc()/free() or new/delete pairs often, it will fragment the heap, causing it to grow, and eventually cause a stack collision. This is not so much about the Arduino being a resource-limited device; it will eventually happen in any continuously running device with a fixed amount of memory.

If you can set an upper bound to the buffer requirements, then pre-allocating a buffer-pool globally, locally as a file-level- or local-static, or on the heap (using malloc() [1]) - an array of "big-enough" buffers and enough of them for your application. Then always allocate/de-allocate one of these and you will avoid fragmentation. Your application (or buffer-handler library) will have to keep track of which buffers are free and which are in use, just as the system allocators do. But in exchange for writing this way, you get to allocate and deallocate memory, forever, fragmentation-free. Every request, and every return is the same size, so any free piece can fill a request with no need to grow the pool.

My personal preference is a static allocation (file-level or globally visible) because 1) it comes from lower memory which has the same effect as allocating it on the heap, and 2) it is declared as an array of arrays so array-indexing syntax works without manually having to write around it as you might if you malloc()ed on big chunk.

Note[1]: Though I have cautioned against using malloc() and free() calls, here and elsewhere on the site, the cause of fragmentation is repeated allocation and de-allocation from/to the heap. In this case, though, I'm suggesting using malloc() to allocate memory which will be kept for the rest of the program run.