What is the memory expense of creating a String from a char array?

Question

I'm writing a little API for processing email messages in Arduinos. Obviously, I need to keep memory use down, but I also want to allow the end user to use the String functions (like indexOf) to search for particular content in emails.

My logic uses char arrays (so that memory is statically allocated). If my end user converts a char array to a String (so that they can easily see if an email subject startsWith() a particular word) will this result in doubling the character memory used? .. or will the String class simply include a pointer to the original char array (and just add some cruft)?

What would be the best-practice in this kind of situation?

Answer 1

What would be the best-practice in this kind of situation?

To not use String.

I also want to allow the end user to use the String functions (like indexOf) to search for particular content in emails.

There's nothing wrong with providing an indexOf function in your API. But you don't have to use String to implement that function when called. You can just use the strstr function on char arrays (reference here).

I should also point out that most Arduinos do not have enough RAM for storing an email.

UPDATE 07-01-2016:

To answer the titular question, the fixed cost of using String is at least 1600 bytes of program space, more if you use more than a few methods. The linker can throw out methods you don't use.

Then for each String variable you declare, the recurring cost is 8 bytes for the variable itself, plus the variable-size memory chunk for your character data. This chunk is allocated from the heap, and is the size of your string plus 4 more bytes to manage the chunk within the heap. That's a total of 12 extra bytes per String instance.

However, I did not originally answer that question because you explicitly asked about the memory expense. This ignores other expenses that should not be ignored:

• execution time - Allocating memory from the heap is really a search for a chunk of the requested size. At the beginning it can be quick, but as the heap gets fragmented (small chunks build up) it takes longer to find a chunk. And when the String is emptied or destroyed, the memory chunk gets put back into the heap and (possibly) merged with adjacent chunks. Reducing the size of a String does not free up any memory. The String maintains its maximum size until it is emptied or destroyed.
• long-term stability - Embedded Systems are supposed run forever. A primary characteristic of a system that can run forever is "deterministic." For whatever inputs it receives, it has a predictable response and end state. When you use the heap via String, the heap gets fragmented according to the data it handles. This is not predictable, and it may get into a state where no memory chunk of the requested size is available. Choke.
• maintenance - because the heap state depends on the processed String data, finding the cause of the failure can be difficult. It can fail at different times because the input data is different, and it can fail at different places in the program.

Because of these expenses, String is rarely used by an experienced developer. Unfortunately, String is tempting to the beginner because it is easy to understand. What is difficult to understand is the complicated, non-deterministic behaviour under the hood that inevitably leads to system failure, rending of garments and gnashing of teeth.

Answer 2

[I'm a year late but this or a related question gets asked often enough, I think it's worth another answer.]

With a Mega, there's enough memory to consider doing what you propose if we make some assumptions and modifications:

1. We know or can accept a maximum size of a string, above which, memory allocation will fail, detectably (the allocator returns a NULL pointer and the object handles the failure gracefully);

2. We use a non-fragmenting allocation method - statically allocated buffer pools of one or more fixed sizes - that are allocated and returned intact. No splitting or merging will take place. A given request will be filled with a buffer of the requested size or larger, if one is available.

3. Either the String object is modified to only use the new allocator or the system 'new()' is replaced by it.

The obvious disadvantages are:

• The entire heap will not be eaten up by a fragmented memory pool. ("Heap" is that space between the statically allocated memory and dynamically growing and shrinking stack. When the stack and heap meet, your application fails).
• You'll need to know something about your application's memory needs to plan the buffer pools.
• You'll have to plan to prevent or handle allocation failure.

In exchange, you get:

• The allocator won't crash your system.
• The String object's programming advantages become available.
• You'll write better code since you had to think about and plan for its needs rather than let a dumb and fragmenting allocator blindly consume the heap.