How do arrays work?

Question

It's been days now I'm trying to figure out how arrays work in Arduino IDE. For better understanding, I assembled some pieces of code I found here and there to write this sketch:

#define arrayLength(x) (sizeof(x) / sizeof(int)) //array size

typedef struct {
  int note;
  int duration;
} BUZZER_NOTE ;

class Buzzer {

  public:

    BUZZER_NOTE notes[];  //-- Funny how the same declaration tried below DOES work if it's inside a class

};


void setup() {

  Serial.begin(9600);

//  BUZZER_NOTE notes[];  //-- Compilation error: 

  int *bob;               //-- Declaring this way, shows no compilation errors, but prints out strange values
//  int bob[];            //-- Compilation error: "storage size of 'bob' isn't known"

  int bob1[] = {10, 20};
  int bob2[5] = {10, 20, 30, 40, 50};
  int bob3[] = {60, 70, 80};

  bob[0] = 10;
  bob[1] = 20;
  bob[2] = 30;
  bob[3] = 40;
  bob[4] = 50;
  bob[5] = 60;
  bob[6] = 70;   //-- No matter how many indexes I create, it will always show lenght of 2

  test(1);       //-- Calls the first function below, as expected
  test({1});     //-- Also calls the first function below, when the second one (with the array parameter) is expected

  test(bob);     //-- Length: 1 (expected 5); Size: 2 (?????) ; First Element: 163 (expected 10); 
  test(bob1);    //-- Length: 1 (expected 2); Size: 2 (?????) ; First Element: 10  (ok); 
  test(bob2);    //-- Length: 1 (expected 5); Size: 2 (?????) ; First Element: 10  (ok); 
  test(bob3);    //-- Length: 1 (expected 3); Size: 2 (?????) ; First Element: 60  (ok); 

}

void loop() {
}

void test(int a) {
  Serial.println("Called int function");
  Serial.println();
}

void test(int a[]) {
  Serial.println("Called array function");
  Serial.print("Length: ");
  Serial.println(arrayLength(a));
  Serial.print("Size: ");
  Serial.println(sizeof(a));
  Serial.print("First Element: ");
  Serial.println(a[0]);
  Serial.println();
}

The code above prints out these lines:

Called int function    // as expected

Called int function    // expected array function

Called array function  // as expected
Length: 1              // expected 5
Size: 2                
First Element: 163     // expected 10

Called array function  // as expected
Length: 1              // expected 5
Size: 2
First Element: 10      // as expected

Called array function  // as expected
Length: 1              // expected 3
Size: 2
First Element: 10      // as expected

Called array function  // as expected
Length: 1              // expected 5
Size: 2
First Element: 60      // as expected

My goal for now is to achieve something like this:

int bob[]; // this gives a compiler error: "storage size of 'bob' isn't known"
bob = {10, 20, 30, 40};

test(bob);

void test(int arr[]) {
  for (int i = 0; i < FIND_ARR_LENGTH_SOMEHOW; i++) {
    Serial.print("Index i: ");
    Serial.println(arr[i]);
}

Question: How can I declare an array without mention its size, fill it up with as many indexes I want, then pass it to a function that will be able to process all the contents?

I know I could declare an array and keep its length to pass as a second parameter, but there must be a way to get an array length.

Thanks in advance.

Answer 1

BUZZER_NOTE notes[];  //-- Funny how the same declaration tried below DOES work if it's inside a class

It doesn't really. C++ language prohibits that. It is just a non-standard quirk of GCC compiler used by Arduino IDE, which happens to support C-style flexible array members in C++ code.

//  BUZZER_NOTE notes[];  //-- Compilation error: 

Of course. You are not allowed to use incomplete types in object definitions. And an array of unspecified size is an incomplete type. A widely used exception from this rule is an [] array declared with an initializer. In that case the array size is assumed to be implicitly specified by the initializer.

int *bob;               //-- Declaring this way, shows no compilation errors, but prints out strange values

Of course. This is not an array at all. It is a pointer. And it doesn't point anywhere. It is invalid. You are not allowed to even access such pointer.

//  int bob[];            //-- Compilation error: "storage size of 'bob' isn't known"

See above.

 int bob1[] = {10, 20};
 int bob2[5] = {10, 20, 30, 40, 50};
 int bob3[] = {60, 70, 80};

These are properly declared arrays. As I said above, the [] declaration is allowed since you are declaring your array with an initializer.

bob[6] = 70;   //-- No matter how many indexes I create, it will always show lenght of 2

This does not make any sense at all. You cannot "create indexes" in an array. Array size is fixed at the point of declaration. There's no way to shrink or expand it, there's no way to "create" or "remove" indexes.

But in this example bob is not even an array. It is a pointer. And since it is a garbage pointer that doesn't point anywhere, you are not allowed to access it with any indexes.

Once you make your bob to point to some valid block of memory, you will be able to access it as bob[3] and such, as long as you stay withing the bounds of that block of memory.

test({1});     //-- Also calls the first function below, when the second one (with the array parameter) is expected

Several mistakes here.

Firstly, you second function does not have "array parameter". In parameter declarations int a[] is equivalent to int *a. It is a pointer parameter, not an array parameter.

Secondly, since the beginning of times in C and C++ you could use a scalar value in {} to initialize a scalar object

int a = { 42 }; /* <- this has always been legal in C and C++ */

So, the first function is obviously a candidate in this case. The rules of C++ language dictate that {1} initializer can be used as an initializer for an int value or std::initializer_list<int> object, but for a pointer. Which is why the first function is called.

You can force it to produce a temporary array through array-type alias and functional-cast syntax

using ARR = int[];
test(ARR{1});

but in GCC you will run into a GCC bug preventing you from forming pointers and references to temporary arrays (Clang will accept it).

void test(int a[]) {
  ...
  Serial.print("Length: ");
  Serial.println(arrayLength(a));

This makes no sense. This "array length" trick

#define arrayLength(x) (sizeof(x) / sizeof(int))

only works with arrays. Your a inside test is not an array. It is a pointer. It is completely pointless to apply this sizeof trick to a pointer.

My goal for now is to achieve something like this:

int bob[]; // this gives a compiler error: "storage size of 'bob' isn't known"
bob = {10, 20, 30, 40};

Not possible. When defining an array, you always have to specify a fixed compile-time size. And arrays in C and C++ are not copyable, not assignable. If you want to copy data between arrays, you have to either do it manually, element by element, or use a library-provided routine that does the same.

{}- initializers can be used with arrays, but only at the point of declaration. Not later. You can do something like

using A = int[5];
A a;
std::memcpy(a, A{1, 2, 3, 4, 5}, sizeof(A));

but this will run into the same GCC bug I mentioned above.

On a related note: standard library provides std::array template - a thin wrapper for raw arrays. std::array objects are copyable and assignable through a pretty much the same syntax as you used

std::array<int, 5> a;
a = { 1, 2, 3, 4, 5 };

But they are still arrays of fixed compile-time size.

void test(int arr[]) {
  for (int i = 0; i < FIND_ARR_LENGTH_SOMEHOW; i++) { 

Not possible. Your arr is a pointer. It is not possible to extract "array size" information for a pointer, since that information simply does not exist. If you want to "FIND_ARR_LENGTH_SOMEHOW" you have to either develop your own convention for finding that length or simply pass that length to the function through a separate parameter.

Answer 2

This is more of a general programming question, but since there are some special pitfalls for microcontrollers/Arduinos I will answer the question.

How can I declare an array without mention its size, fill it up with as many indexes I want, then pass it to a function that will be able to process all the contents?

Short answer: You cannot. An array has always a fixed size, that cannot be changed during the lifetime of the array.

Long answer: Though every array has it's fixed size, you can create new arrays with a size, that is not known at compilation time. For this you would need dynamic memory allocation like done with the new keyword:

BUZZER_NOTE *notes = new BUZZER_NOTE[x];

with the size of the new array x. notes is here a pointer to the array (actually a pointer to the first element of the array). If you don't know, how to deal correctly with pointers, you should google for a tutorial about it, there are plenty on the web. So, if your defined array is full and you want to add more elements, you have to define a new array with bigger size, copy the old content and delete the old one (actually that is what happens inside the String class).

To actually get the easy way of adding more elements, without copying the whole content, a C++ programmer would use a linked list. This defines an element class, that contains a variable for the content of the element and a pointer to the next element in the list. So to iterate through the linked list, you would start at the first element and follow the pointers through the list to the end. (Again you can read more about this concept on the web)

BUT: You should not use any of these things on an Arduino, especially not on the ones with very limited RAM (like Uno, Nano, ...). When you do dynamic memory allocation, memory fragmentation will happen. Take the example from above. You have a dynamically allocated array with your data. Now you want to store more data. You define a new and bigger array. For the simplicity of explanation, it will be allocated directly after the first array in the memory. You copy the content and delete the first array to free the memory. Now there is free space in your memory with the size of the first array. Now you want to again add more data, so you define a new and bigger array. This array cannot fit into the free space at the start, since the space to not big enough. This memory space is now dead, because it cannot be used for your data.

In real cases, you would allocate and delete variables with varying sizes throughout your sketch. This leads to the point, that you will get many holes in your memory, which are too small to fit your data in. On a PC this is not a big problem, since you have a lot of RAM and the OS can reorganize the data to fit better. But on the Arduino you have only very limited RAM and no OS, so you quickly run out of RAM. Your sketch will stop to work at that point, since there is no usable memory left on the device.

You instead should go the harder way and just use fixed arrays, that are big enough to hold the biggest amount of data, that you expect.

---

And since AnT didn't cover this completely:

bob[6] = 70;

This line cannot work correctly for 2 reasons: First you only declared bob as a pointer, but this pointer does not point to a valid memory address. You would have to create a corresponding array with the new keyword as shown above, or let it point to an existing array. Currently you try to write to an invalid or random memory address.

Secondly the compiler cannot know, how many items the array has, that bob should point to. This is just not an information, that the compiler has. So it will not prevent you from writing higher indices, than are in the array. But effectively the memory write either fails, because the address of the non-existent element is invalid, or you will overwrite data from some other variables --> Very dangerous! It is your responsibility to ensure, that you don't write data, where it does not belong.

And keep in mind, that microcontrollers don't have something like exceptions. Thus some things, that would through an exception on a PC will stop the microcontroller from working, or it will just do the crazy stuff, that you told him to do (where accessing memory outside of the array would through a runtime error on the PC).

Answer 3

Ok, after reading and reading the answers and links about my question, I ended up using a function as suggested. Something like:

void foo(char* myArray, int size) {}

But I ended up finding another solution to the problem, which is a little more complicated:

#define arrayLen(x) (sizeof(x) / sizeof(x[0]))

char myArray[64] = "my array test with a long value in it";

void setup() {

  Serial.begin(9600);
  while (!Serial);

  // IMHO, it would be nice if the compiler would accept these two sintaxes
//  myArray = "my array test with a long value in it";
//  "my array test with a long value in it".toCharArray(myArray, 64);

  Serial.print("Value before: ");
  Serial.println(myArray);
  Serial.print("Size before: ");
  Serial.println(arrayLen(myArray));

  String temp;
  temp = myArray;

  foo(temp);

  Serial.print("Value after: ");
  Serial.println(myArray);
  Serial.print("Size after: ");
  Serial.println(arrayLen(myArray));

}

void loop() {}

void foo(String str) {
  for (int i = 0; i < str.length(); i++) {
    Serial.print("Index ");
    Serial.print(i);
    Serial.print(": ");
    Serial.println(str.charAt(i));
  }
}

I know I have a bad habit to use Strings instead of array of chars (coming from other languages), but in this example, memory limitation issues won't be significant because the String will be freed from memory after execution of setup() and foo().

There is another thing I would like to share (not directly related to my previous question), that I believe could be useful for other people. This little sketch shows how to use the same String technique to manipulate the contents of a char array, passed by reference to an external function. Here it goes:

#define arrayLen(x) (sizeof(x) / sizeof(x[0]))

char myArray[64] = "my array test with a long value in it";

void setup() {

  Serial.begin(9600);
  while (!Serial);

  Serial.print("Value before: ");
  Serial.println(myArray);
  Serial.print("Size before: ");
  Serial.println(arrayLen(myArray));

  String temp;
  temp = myArray;

  foo(temp);
  temp.toCharArray(myArray, 64);

  Serial.print("Value after: ");
  Serial.println(myArray);
  Serial.print("Size after: ");
  Serial.println(arrayLen(myArray));

}

void loop() {}

void foo(String &str) {
  str = "abcde";
}

Results (just as expected):

Value before: my array test with a long value in it
Size before: 64
Value after: abcde
Size after: 64

It may look awful to the eyes of a C++ programmer, but hey, it works!

Thanks again for all the answers/comments.

Answer 4

The C arrays are tricky and they're described in every C tutorial. The other answers adresses mistakes / questions in comments, but none of them shows how it can be done:

struct BuzzNote {
  int note;
  int duration;
};

const BuzzNote melody[] = {
  {10, 200}, { 45, 300}, {90, 200}, // ...
};

void test(BuzzNote const * arr, size_t len) {
  for (size_t i = 0; i < len; ++i) {
    Serial.print(i);
    Serial.print(". ");
    Serial.print(arr[i].note);
    Serial.print(' ');
    Serial.println(arr[i].duration);
  } 
}

// using template for determining size of array
// (and as a bonus it won't work with plain pointers)
template<typename T, size_t size> 
size_t GetArrLength(T(&)[size]){return size;}

void setup() {
  Serial.begin(9600);
}

void loop() {
  test(melody, GetArrLength(melody));
  delay(10000);
}

I'm using template version of getting array length (it won't compile with pointers, as pointer isn't array).

And this version stores data in RAM (and waste it). If you don't have spare space in ram, you'll have to dig into PROGMEM variables/arrays. But it's the quest for another day :D