Maximum Capacity Debugging

Question

1. Example

Check the program below for an Arduino Nano, Old Bootloader, for variable values of N.

• For low values (N<=1857), the program outputs "[FFFFFFFF]",
• For higher values (1858<=N<=1953), the program outputs unpredictable variations of this string,
• For higher values (1954<=N<=1955), the program outputs fails flooding the symbol "[",
• Over a certain value (1956<=N), the programs does not output any character.

I didn't check extensively these results, which could be very different actually, and totally platform dependent, but I guess, reproducible.

Note that no dynamic allocation is being used.

Note that everything is initialized, first 4 values to 0xFF, next N-3 values to 0x00 (Ref.).

The compilation do not show any relevant alert, showing a modest 10% capacity usage.

Sketch uses 3206 bytes (10%) of program storage space. Maximum is 30720 bytes.
Global variables use 196 bytes (9%) of dynamic memory, leaving 1852 bytes for local variables. Maximum is 2048 bytes.

It is evident from this example how to figure the proper value of N, indeed a capacity related issue could be expected (Ref.).

Note these values are by far less than the "guess" value of N=32767 for 1 byte variables (?) (Ref.).

2. Problem

Now, with a considerably huge, real program, I am having the same issue regarding those unexpected results, so I am almost sure (few moments ago) than this is the cause.

When exceeding some "~maximum~" level, the program starts failing, and when "~releasing~" this capacity, the program starts working well (without those deleted functionality of course!).

3. Question

How should I debug, measure, dimension, or prevent this situation, and specify when it is the proper time to keep this processor, and when should I move into a bigger one, when clearly the compiler is not a reasonable guess?.

// CAPACITY TEST
#include <SoftwareSerial.h>

// SET THE PROPER N VALUE
// N<=1857: Last Valid Result
// 1858<=N<=1953: Last Observable Output - Single Char
//     PC=0x0444. [AVR MEMORY] Writing to memory location 0x0948 outside of memory size 0x0900. [UP0]
// 1954<=N<=1955: Last Observable Output - Flood
//     Invalid opcode 0xFFFF at PC=0xA402
//     PC=0x0444. [AVR MEMORY] Writing to memory location 0x0948 outside of memory size 0x0900. [UP0]
// 1956<=N: No Output
//     Invalid opcode 0xFFFF at PC=0xA402
#define N 1857

void setup()
{
    Serial.begin(38400);while (!Serial){;}

    byte a[N+1]={0xFF,0xFF,0xFF,0xFF};

    char si[3];
    Serial.print("[");
    for (int i1=0;i1<=N-1&&a[i1]!=0;i1++){
        sprintf(si,"%02X",a[i1]);
        Serial.print(si);
    }
    Serial.print("]");
}

void loop(){}

Answer 1

From the bottom to the top of the RAM, you have:

• the .data section (statically allocated initialized variables)
• the .bss section (statically allocated non initialized variables)
• the heap (dynamic allocations with malloc() and new)
• free space
• the stack (return addresses and automatic local-variable allocations)

See the memories of an Arduino for a more lengthy discussion.

The sizes of the .data and .bss sections are known at build time. This is the “Global variables” memory usage reported at the end of the compilation. The rest of the RAM is available for the heap and the stack. This is the amount reported as “leaving 1852 bytes for local variables”, where “local variables” is a slight oversimplification.

Your example program doesn't use heap allocation. It does use some static allocation, mostly for the Serial buffers, and a lot of stack for:

• the return addresses from the calls to main() and setup() (4 bytes)
• the local arrays a and si (N+4 bytes)
• the memory needed by sprintf() or Serial.print() (unknown)

Even ignoring the last item, you can see that the whole RAM will be consumed when 4 + N + 4 = 1852, i.e. N = 1844. As you make N larger than this value, the stack bumps into the .bss section, and you start corrupting your global variables. The behavior you describe is consistent with the program corrupting the internal data structures of the Serial object.

How should I debug, measure, dimension, or prevent this situation

There is no easy way to determine beforehand the amount of heap and stack your program will consume. The usual solution is to measure and report it at run time. See for example the MemoryFree library (there are many similar to this).