0

I know that with 8-bit AVR Arduinos, std::string is widely regarded as fundamentally unsafe due to problems with heap-fragmentation (partly due to limited ram, and partly because the WAY the underlying Arduino/AVR library manages memory is more simplistic than something constantly allocating and releasing a lot of ephemeral allocations needs to be).

Does the same caution apply to ESP32 (when used with Arduino framework), or does its underlying memory-management scheme (presumably based upon esp-idf) on a module with ~468k of RAM do a sufficiently better job of minimizing fragmentation and memory leaks to make std::string (and other similar classes) safe to use without having to constantly obsess about the problem and bend over backwards to avoid ephemeral memory allocations?

I'm particularly worried that the various libraries for making http requests (potentially involving the creation and destruction of multiple String objects for every request and response) might be more indifferent to the problem than they really ought to be.

I know there are a few Arduino libraries floating around that basically allocate one single big chunk of memory for a reusable String-like object (and additional space to give it "working room" to do things like regular expressions, substring replacements, etc), but it looks like the popular ESP32 libraries for making http requests just create and destroy new String objects for every request and act like it's totally fine. Am I being paranoid, or is ephemeral memory allocation still a real problem, even on platforms like ESP32?

By the same token... is there some way (on an ESP32 Arduino program) to periodically ask the library to analyze the current memory situation, and deliberately initiate a software-reboot (and re-initialization) if the memory appears to be getting overly fragmented (vs waiting for a slow memory leak to just fester until a memory allocation fails and the whole program crashes uncontrollably)?

2 Answers 2

0

The problem with randomly allocating and deallocating memory in embedded systems (ES) is that most are meant to be able to run error free, essentially forever. Since memory space is finite, an ES that employs the technique don't meet that constraint.

To the question in your last paragraph, apparently you are considering a system that can tolerate occasional resets with the consequent loss of context. In that case, perhaps resetting anytime an allocation fails would be acceptable. You'd want to vet each request especially carefully to to ensure it could usually be expected to succeed (define "usually"!), to avoid the large overhead of too-frequent resets or worse, a continuous reset loop resulting from an impossible-to-fill request.

Consider the difference between a human-driven question-and-answer system on one hand, and an implanted cardiac pacemaker or an jet fighter auto-landing system for night carrier-landings. An occasional timing hiccup might be annoy me in the Q&A system (if I can even notice it), but in either of the other two, it might spoil someone's whole day. :-(

2
  • The system could definitely tolerate periodic reboots without incident, as long as they happen when I decide they're necessary (say, upon checking as the final act in loop(), and triggering a reboot if the largest chunk of free memory is smaller than 100k, or less than 200k remains available). The general assumption is that it wouldn't happen very often... but from what I've seen, Arduino-based stuff in general that uses wifi for almost anything tends to reliably crap out after a few weeks unless it gets periodically rebooted... so I'm just starting from that assumption and embracing it :)
    – Bitbang3r
    Commented Jan 22 at 1:06
  • For what it's worth, I have a fan controller on an ATmega328p -> ESP-01 (AT firmware) that runs continuously until a power glitch or some other event takes down my WiFi network. If it were more critical to keep the controller running, it could have been made to "don't care" if the WiFi is down and just keep going without it. The WiFi is only to allow me to view what the controller is doing and isn't a necessary function.
    – JRobert
    Commented Jan 22 at 13:19
0

If this is any interest to you, I did a web server library that doesn't use String class at all.

It decodes incoming HTTP requests an item at a time, passing the results to a callback routine.

That way it doesn't even have to hold the whole HTTP request in memory, it decodes the individual parameters "on the fly" using a state machine.

Since there is no dynamic memory allocation you should, in theory, be able to run indefinitely.

My library is at https://github.com/nickgammon/HTTPserver

Having said that, and to answer your actual question ...


Dynamic memory allocation doesn't have to be bad. In particular, if the amounts of memory being allocated are always the same (for example, your input to your server is predictable) then it shouldn't run out of memory. The allocator will just keep serving up the same bit of memory you used last time.

What can be particularly bad is this sort of stuff:

String buf = "";
while (Serial.available ())
  {
  buf += Serial.read ();  // append one byte
  }

That sort of code is likely to cause heap fragmentation because it keeps allocating larger and larger chunks. Whether the library you mention does that, I don't know.


Note, it isn't totally clear from your question if you are writing an HTTP client or server. If you are writing a client then my library won't be any help, but my remarks above should still apply.

3
  • Thanks, but my program is purely a http client. The main thing I'm wringing my hands over is that the circuit's primary mission is, "monitor moisture sensors, and trigger the piezo buzzer if water is detected". I have it sending BLE advertisements for my debugging convenience, and making occasional http POST requests to log the data & make it available to me from outside the house... but my big concern is making sure that failure with Bluetooth or WiFi won't hang or crash the whole program and derail it from its primary mission (of watching the sensors & sounding the piezo if necessary).
    – Bitbang3r
    Commented Jan 28 at 1:47
  • (continued)... I think I might need to eventually implement something involving watchdog timers... but I'm not sure whether ESP32 even has AVR-like watchdog timers capable of automatically rebooting the chip if the software fails to "check in" within some specified time limit. In any case, that would be a topic for another question ;-)
    – Bitbang3r
    Commented Jan 28 at 1:51
  • According to a Google search, it has in fact got watchdog timers.
    – Nick Gammon
    Commented Jan 28 at 7:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.