50

The use of malloc() and free() seems pretty rare in the Arduino world. It is used in pure AVR C much more often, but still with caution.

Is it a really bad idea to use malloc() and free() with Arduino?

  • 2
    you'll run out of memory really fast otherwise, and if you know how much memory you'll use you might as well statically allocate it anyway – ratchet freak Mar 9 '14 at 13:32
  • 1
    I don't know if it's bad, but I think it isn't used because you almost never run out of RAM for most sketches and it's just a waste of flash and precious clock cycles. Also, don't forget about scope (although I don't know if that space is still allocated for all variables). – Anonymous Penguin Mar 9 '14 at 16:50
  • 4
    As usual, the right answer is "it depends." You haven't provided enough information to know for sure whether dynamic allocation is right for you. – WineSoaked Mar 9 '14 at 17:22
40

My general rule for embedded systems is to only malloc() large buffers and only once, at the start of the program, e.g., in setup(). The trouble comes when you allocate and de-allocate memory. Over a long run session, memory becomes fragmented and eventually an allocation fails due to lack of a sufficiently large free area, even though the total free memory is more than adequate for the request.

(Historical perspective, skip if not interested): Depending on the loader implementation, the only advantage of run-time allocation vs. compile-time allocation (intialized globals) is the size of the hex file. When embedded systems were built with off the shelf computers having all volatile memory, the program was often uploaded to the embedded system from a network or an instrumentation computer and the upload time was sometimes an issue. Leaving out buffers full of zeros from the image could shorten the time considerably.)

If I need dynamic memory allocation in an embedded system, I generally malloc(), or preferably, statically allocate, a large pool and divide it into fixed-size buffers (or one pool each of small and large buffers, respectively) and do my own allocation/de-allocation from that pool. Then every request for any amount of memory up to the fixed buffer size is honored with one of those buffers. The calling function doesn't need to know whether it's larger than requested, and by avoiding splitting and re-combining blocks we solve fragmentation. Of course memory leaks can still occur if the program has allocate/de-allocate bugs.

  • Another historical note, this quickly led to the BSS segment, which allowed a program to zero its own memory for initialization, without slowly copying the zeros during program load. – rsaxvc Apr 13 '16 at 5:22
16

Typically, when writing Arduino sketches, you will avoid dynamic allocation (be it with malloc or new for C++ instances), people rather use global -or static- variables, or local (stack) variables.

Using dynamic allocation can lead to several problems:

  • memory leaks (if you lose a pointer to a memory you previously allocated, or more likely if you forget to free the allocated memory when you don't need it anymore)
  • heap fragmentation (after several malloc/free calls) where the heap grows bigger thant the actual amount of memory allocated currently

In most situations I have faced, dynamic allocation was either not necessary, or could be avoided with macros as in the following code sample:

MySketch.ino

#define BUFFER_SIZE 32
#include "Dummy.h"

Dummy.h

class Dummy
{
    byte buffer[BUFFER_SIZE];
    ...
};

Without #define BUFFER_SIZE, if we wanted Dummy class to have a non-fixed buffer size, we would have to use dynamic allocation as follows:

class Dummy
{
    const byte* buffer;

    public:
    Dummy(int size):buffer(new byte[size])
    {
    }

    ~Dummy()
    {
        delete [] bufer;
    }
};

In this case, we have more options than in the first sample (e.g. use different Dummy objects with different buffer size for each), but we may have heap fragmentation issues.

Note the use of a destructor to ensure dynamically allocated memory for buffer will be freed when a Dummy instance is deleted.

15

I have taken a look at the algorithm used by malloc(), from avr-libc, and there seems to be a few usage patterns that are safe from the point of view of heap fragmentation:

1. Allocate only long-lived buffers

By this I mean: allocate all you need at the beginning of the program, and never free it. Of course, in this case, you could as well use static buffers...

2. Allocate only short-lived buffers

Meaning: you free the buffer before allocating anything else. A reasonable example might look like this:

void foo()
{
    size_t size = figure_out_needs();
    char * buffer = malloc(size);
    if (!buffer) fail();
    do_whatever_with(buffer);
    free(buffer);
}

If there is no malloc inside do_whatever_with(), or if that function frees whatever it allocates, then you are safe from fragmentation.

3. Always free the last allocated buffer

This is a generalization of the two previous cases. If you use the heap like a stack (last in is first out), then it will behave like a stack and not fragment. It should be noted that in this case it is safe to resize the last allocated buffer with realloc().

4. Always allocate the same size

This will not prevent fragmentation, but it is safe in the sense that the heap will not grow larger than the maximum used size. If all your buffers have the same size, you can be sure that, whenever you free one of them, the slot will be available for subsequent allocations.

  • 1
    Pattern 2 should be avoided as it adds cycles for malloc() and free() when this can be done with "char buffer[size];" (in C++). I would also like to add the anti-pattern "Never from an ISR". – Mikael Patel Feb 1 '16 at 14:50
9

Using dynamic allocation (via malloc/free or new/delete) isn't inherently bad as such. In fact, for something like string processing (e.g. via the String object), it's often quite helpful. That's because many sketches use several small fragments of strings, which eventually get combined into a larger one. Using dynamic allocation lets you use only as much memory as you need for each one. In contrast, using a fixed-size static buffer for each one could end up wasting a lot of space (causing it to run out of memory much faster), although it depends entirely on the context.

With all of that being said, it's very important to make sure memory usage is predictable. Allowing the sketch to use arbitrary amounts of memory depending on run-time circumstances (e.g. input) can easily cause a problem sooner or later. In some cases, it might be perfectly safe, e.g. if you know the usage will never add up to much. Sketches can change during the programming process though. An assumption made early-on could be forgotten when something is changed later, resulting in an unforeseen problem.

For robustness, it's usually better to work with fixed-size buffers where possible, and design the sketch to work explicitly with those limits from the outset. That means any future changes to the sketch, or any unexpected run-time circumstances, should hopefully not cause any memory problems.

6

I disagree with people who think you shouldn't use it or it is generally unnecessary. I believe it can be dangerous if you don't know the ins and outs of it, but it is useful. I do have cases where I don't know (and shouldn't care to know) the size of a structure or a buffer (at compile time or run time), especially when it comes to libraries I send out into the world. I agree that if you're application is only dealing with a single, known structure, you should just bake in that size at compile time.

Example: I have a serial packet class (a library) that can take arbitrary length data payloads (can be struct, array of uint16_t, etc.). On the sending end of that class you simply tell the Packet.send() method the address of the thing you wish to send and the HardwareSerial port through which you wish to send it. However, on the receiving end I need a dynamically allocated receive buffer to hold that incoming payload, as that payload could be a different structure at any given moment, depending on the application's state, for instance. IF I'm only ever sending a single structure back and forth, I'd just make the buffer the size it needs to be at compile time. But, in the case where packets may be different lengths over time, malloc() and free() are not so bad.

I've run tests with the following code for days, letting it loop continuously, and I've found no evidence of memory fragmentation. After freeing the dynamically allocated memory, the free amount returns to its previous value.

// found at learn.adafruit.com/memories-of-an-arduino/measuring-free-memory
int freeRam () {
    extern int __heap_start, *__brkval;
    int v;
    return (int) &v - (__brkval == 0 ? (int) &__heap_start : (int) __brkval);
}

uint8_t *_tester;

while(1) {
    uint8_t len = random(1, 1000);
    Serial.println("-------------------------------------");
    Serial.println("len is " + String(len, DEC));
    Serial.println("RAM: " + String(freeRam(), DEC));
    Serial.println("_tester = " + String((uint16_t)_tester, DEC));
    Serial.println("alloating _tester memory");
    _tester = (uint8_t *)malloc(len);
    Serial.println("RAM: " + String(freeRam(), DEC));
    Serial.println("_tester = " + String((uint16_t)_tester, DEC));
    Serial.println("Filling _tester");
    for (uint8_t i = 0; i < len; i++) {
        _tester[i] = 255;
    }
    Serial.println("RAM: " + String(freeRam(), DEC));
    Serial.println("freeing _tester memory");
    free(_tester); _tester = NULL;
    Serial.println("RAM: " + String(freeRam(), DEC));
    Serial.println("_tester = " + String((uint16_t)_tester, DEC));
    delay(1000); // quick look
}

I haven't seen any sort of degradation in RAM or in my ability to allocate it dynamically using this method, so I'd say it's a viable tool. FWIW.

  • 2
    Your test code conforms to the usage pattern 2. Allocate only short-lived buffers I described in my previous answer. This is one of those few usage patterns known to be safe. – Edgar Bonet May 18 '15 at 15:33
  • In other words, the problems will come up when you start sharing the processor with other unknown code - which is precisely the problem you think you are avoiding. Generally, if you want something that will always work or else fail during linking, you make a fixed allocation of the maximum size and use it over and over again, for example by having your user pass it in to you in initialization. Remember you are typically running on a chip where everything has to fit in 2048 bytes - maybe more on some boards but also maybe a lot less on others. – Chris Stratton May 18 '15 at 16:48
  • @EdgarBonet Yes, exactly. Just wanted to share. – StuffAndyMakes May 19 '15 at 17:29
  • 1
    Dynamically allocating a buffer of only the size needed is risky, as if anything else allocates before you free it you can be left with fragmentation - memory that you can't re-use. Also, dynamic allocation has tracking overhead. Fixed allocation doesn't mean you can't multiply use the memory, it just means that you have to work the sharing into the design of your program. For a buffer with purely local scope, you might also weigh use of the stack. You haven't checked for the possibility of malloc() failing either. – Chris Stratton May 19 '15 at 17:46
  • 1
    "it can be dangerous if you don't know the ins and outs of it, but it is useful." pretty much sums up all development in C/C++. :-) – ThatAintWorking Nov 16 '17 at 17:19
4

Is it a really bad idea to use malloc() and free() with Arduino?

The short answer is yes. Below are the reasons why:

It is all about understanding what an MPU is and how to program within the constraints of the available resources. The Arduino Uno uses an ATmega328p MPU with 32KB ISP flash memory, 1024B EEPROM, and 2KB SRAM. That is not a lot of memory resources.

Remember that the 2KB SRAM is used for all global variables, string literals, stack and possible usage of the heap. The stack also needs to have head room for an ISR.

The memory layout is:

SRAM map

Todays PC/laptops have more than 1.000.000 times the amount of memory. A 1 Mbyte default stack space per thread is not uncommon but totally unrealistic on an MPU.

An embedded software project has to do a resource budget. This is estimating ISR latency, necessary memory space, compute power, instruction cycles, etc. There are unfortunately no free-lunches and hard real-time embedded programming is the most difficult of programming skills to master.

  • Amen to that: "[H]ard real-time embedded programming is the most difficult of programming skills to master." – StuffAndyMakes Nov 17 '17 at 17:52
  • Is the execution time of malloc always the same? I can imagine malloc taking more time as it searches further in the available ram for a slot that fits? This would be yet another argument (aside running out of ram) to not allocate memory on the go? – Paul May 9 at 22:52
  • @Paul The heap algorithms (malloc and free) are typically not constant execution time, and not reentrant. The algorithm contains search and data structures that require locks when using threads (concurrency). – Mikael Patel May 10 at 9:06
0

Ok, I know this is an old question but the more I read through the answers the more I keep coming back to an observation that seems salient.

The Halting Problem Is Real

There seems to be a link with Turing's Halting Problem here. Allowing dynamic allocation increases the odds of said 'halting' so the question becomes one of risk-tolerance. While it is convenient to wave off the possibility of malloc() failing and so forth, it is still a valid outcome. The question the OP asks only appears to be about technique, and yes the details of the libraries used or the specific MPU do matter; the conversation turns towards reducing the risk of the program halting or any other abnormal end. We need to recognize the existence of environments that tolerate risk vastly differently. My hobby project to display pretty colors on a LED-strip won't kill someone if something unusual happens but the MCU inside a heart-lung machine likely will.

Hello Mr. Turing My Name is Hubris

For my LED-strip, I don't care if it locks up, I'll just reset it. If I were on a heart-lung machine controlled by an MCU the consequences of it locking up or failing to operate are literally life and death, so the question about malloc() and free() should be split between how the intended program deals with the possibility of demonstrating Mr. Turing's famous problem. It can be easy to forget that it is a mathematical proof and to convince ourselves that if only we are clever enough we can avoid being a casualty of the limits of computation.

This question should have two accepted answers, one for those who are forced to blink when staring The Halting Problem in the face, and one for all others. While most uses of the arduino are likely not mission critical or life-and-death applications, the distinction is still there regardless of which MPU you may be coding.

  • I don't think the Halting problem applies in this specific situation considering the fact that heap usage isn't necessarily arbitrary. If used in a well-defined manner then heap usage becomes predictably "safe". The point of the Halting problem was figure out if it can be determined what happens to a necessarily arbitrary and not-so-well-defined algorithm. It really applies much more to programming in a broader sense and as-such I find it to be specifically not very relevant here. I don't even think it's relevant at all to be entirely honest. – Jonathan Gray Sep 2 at 11:50
  • I'll admit to some rhetorical exaggeration but the point is really if you want to guarantee behavior, using the heap implys a level of risk that is much higher than sticking to only using the stack. – Kelly S. French Sep 2 at 18:34
-3

No, but they must be used very carefully in regards to free()ing allocated memory. I have never understood why people say direct memory management should be avoided as it implies a level of incompetence thats generally incompatible with software development.

Lets say your using your arduino to control a drone. Any error in any part of your code could potentially cause it to fall out of the sky and hurt someone or something. In other words, if someone lacks the competences to use malloc, they likely shouldn't be coding at all since there are so many other areas where small bugs can cause serious issues.

Are the bugs caused by malloc harder to track down and fix? Yes, but thats more of a matter of frustration on the coders part rather than risk. As far as risk goes, any part of your code can be equally or more risky than malloc if you don't take the steps to make sure it's done right.

  • 4
    It's interesting you used a drone as an example. According to this article (mil-embedded.com/articles/…), "Due to its risk, dynamic memory allocation is forbidden, under the DO-178B standard, in safety-critical embedded avionics code." – Gabriel Staples Oct 5 '15 at 0:30
  • DARPA has a long history of allowing contractors to develop specs that fit their own platform - why shouldn't they when it's tax payers that pay the bill. This is why it costs $10 billion for them to develop what others can do with $10,000. Almost sounds as if your using the military industrial complex as an honest reference. – JSON May 15 '18 at 1:21
  • Dynamic allocation seems like an invitation for your program to demonstrate the limits of computation described in the Halting Problem. There are some environments that can handle a small amount of risk of such halting and there do exist environments (space, defense, medical, etc.) that will not tolerate any amount of controllable risk, thus they disallow operations that "shouldn't" fail because 'it should work' isn't good enough when you are launching a rocket or controlling a heart / lung machine. – Kelly S. French May 9 at 15:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.