0

In this link, section "Memory, memory, memory" it's been told that doing many String concatenations in sequence, e.g.: using operator+() multiple times, will cause memory fragmentation, which may lead to the fact that total heap not being totally used.

An alternative I'm suggesting is using strcat() with char buffers and not using Strings at all, and then convert from resultant char buffer to String if a String is needed at the end? Would that be an escape from fragmentation of the heap?

Clarification: I will specify the upper bound of the char buffer beforehand, as if I know the maximum needed/to-be-used size.

2

If you size the buffer to be able to hold your final string, then yes, that method will avoid the fragmentation caused by the String concatenation function.

Note that String holds the string as a C-style char array internally; it is the concatenation procedure which dynamically allocates and deallocates memory, causing the fragmentation.

  • I wonder, if simply by allocating and deallocating then memory is fragmented, a char buffer then causes fragmentation! (I mean since I will be freeing it) But since the allocated size is fit to requirements then it's still a lot better than String concatenation. Agree? – YoussefDir Aug 24 '18 at 13:38
  • What the String object does is allocate a new piece of memory large enough to contain the resulting concatenated data, copies both source strings to the new memory, and then deallocates the old memory (of the first String). This means there is a free bit of memory that is now smaller than any subsequent needs for future concatenation of the string in question, so that bit of free memory could be used again by some other function, but it won't be used again by the String being concatenated onto. – jose can u c Aug 24 '18 at 14:01
  • I think that the the standard C string manipulators are better for small micros than the String object, though they are more complicated to use. Both methods work in many cases, but you must be aware of how they work so you can understand when debugging problems. – jose can u c Aug 24 '18 at 14:02
1

Your method would reduce the fragmentation. But will your application need do this once? A few times? Must it run forever? If your application uses malloc()/free() or new/delete pairs often, it will fragment the heap, causing it to grow, and eventually cause a stack collision. This is not so much about the Arduino being a resource-limited device; it will eventually happen in any continuously running device with a fixed amount of memory.

If you can set an upper bound to the buffer requirements, then pre-allocating a buffer-pool globally, locally as a file-level- or local-static, or on the heap (using malloc() [1]) - an array of "big-enough" buffers and enough of them for your application. Then always allocate/de-allocate one of these and you will avoid fragmentation. Your application (or buffer-handler library) will have to keep track of which buffers are free and which are in use, just as the system allocators do. But in exchange for writing this way, you get to allocate and deallocate memory, forever, fragmentation-free. Every request, and every return is the same size, so any free piece can fill a request with no need to grow the pool.

My personal preference is a static allocation (file-level or globally visible) because 1) it comes from lower memory which has the same effect as allocating it on the heap, and 2) it is declared as an array of arrays so array-indexing syntax works without manually having to write around it as you might if you malloc()ed on big chunk.

Note[1]: Though I have cautioned against using malloc() and free() calls, here and elsewhere on the site, the cause of fragmentation is repeated allocation and de-allocation from/to the heap. In this case, though, I'm suggesting using malloc() to allocate memory which will be kept for the rest of the program run.

  • When setting an upper-bound e.g. char buff[ 100 ], am I not allocating it on the stack? I want to use the heap instead. Because the stack in esp8266 I'm working on is only 4KB if I'm not mistaken. – YoussefDir Oct 27 '18 at 12:49
  • I was not clear that the buffer pool needs to be allocated made statically - that is, as a global, a static, or from the heap (with malloc()). I've updated my answer to clarify that. – JRobert Oct 27 '18 at 14:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.