1

I have a sensor giving me values (ints) which I want to keep, and send to a computer via the WiFi shield when requested via HTTP. I have the HTTP thing working, however, I do not know how to have a String or whatever able to store what is basically an array of ints, when I do not know any maximum size.

I have read that using Strings on Arduino is not recommended because how RAM works and such things, so I translated all my String uses to C-style arrays (as it was recommended), this solved my memory issues, however, C-style strings are not resizable.

While browsing the forums, I found that for similar problems the recommended way of handling things would be to create an array of max size instead of dynamically growing the array, but I do not know the maximum size in this case.

Here are the requirements

  • A way of storing each values (in order) so they can be sent later after an HTTP request (but they can get requested multiple times so I can't delete them then),
  • Whatever storage way is fine by me (Strings, arrays of ints, ...), as long as it works.
  • A request to the data should take ~100ms (I don't think this will cause issues, but I'll mention it just in case).

I thought about printing the values to the SD card, since there I have far enough memory, but for some reason I can't get the WiFi and the SD card to work together (as soon as I init the SD library, the WiFi library doesn't work anymore - I asked an other question about this here).

PS. I'm quite new to C++ and Arduino, but I'm not too bad at Java.

What would be the best (or at least a working one) way of achieving this ?

Improve this question
4
  • If this was an ordinary computer I would suggest using C++ vectors as you don't need to describe its size to the compiler. But we are talking embedded and there may not be enough physical RAM for you to work with. Have you consider using a much more powerful Arduino such as an ARM based Arduino? Also, likely all libraries have never been tested with all other libraries for comparability. Given the limited hardware resources of most Arduino processors, you are likely running into a conflict between the WIFI and SDCard libraries.
    – st2000
    Apr 15, 2017 at 16:50
  • Also, considering the low cost of most Arduino platforms, it may be easer to split the tasks between two Arduinos. The sensor sampling & SDCard logging can be accomplished on the 1st Arduino. And the WIFI & web page serving can be accomplished on the 2nd Arduino. It is certainly not the first time hardware has been used instead of software to solve a problem.
    – st2000
    Apr 15, 2017 at 16:53
  • It would be a great solution, but I saddly can't do this (mostly because this is a project for school and I can't choose the hardware I'm working with, along with the fact that time is beginning to run out...)
    – CLOVIS
    Apr 15, 2017 at 17:15
  • Someone might have a direct solution for you (use the libraries as is and still run on a single Arduino). Otherwise, start thinking "out of the box". Consider polling the Arduino on a periodic bases to avoid storing large amounts of data on the embedded processor. Again, investigate alternative Arduino's with large amounts of built in RAM (such as the ARM based Ardiuno). In this way you can set aside large amounts of RAM for data storage with little modifications to your existing WIFI only code (w/o the SDCard code).
    – st2000
    Apr 15, 2017 at 18:01

2 Answers 2

Reset to default
1

This is basically two questions: how to get your SD card and WiFi working at the same time (in which case memory limitations are not an issue), and how to deal with the memory limitations if you choose in-memory storage of your data. I'll address the second one here; you should split out the WiFi/SD conflict into a separate question where you detail the exact hardware you're using and how it's connected.

You've got only 2 KB of RAM in an Uno and similar models, so you will have to discard data at some point, whether or not it was successfully sent in a reply to a web request. The key here is to use a fairly compact method of storing the data and to have an easy way to get rid of old values to free up space for new values.

One common way to deal with this is to use a ring buffer. This is a fixed size storage array with a pointer or index, which I'll call head, indicating the next storage location to use. You start with head = 0 (the beginning of the buffer) and every time you receive a new value you store it at head and then increment head by one. You also need to compare head to the end of the buffer and, when it would pass the end, reset it back to the beginning of the buffer. After the first pass through the buffer, storing at head will overwrite the oldest value previously stored.

Going back through the buffer to read out the old values is easy: you just start by reading the value at head - 1 (or the end of the buffer if head is 0) and work downwards, wrapping around when you hit 0, and stopping when you reach head.

You can handle the initial states where the buffer hasn't been filled yet in two ways: just fill the buffer with a value such as 0 that you don't mind returning as a "nothing stored here" value, or just set a flag at the start to say that the buffer hasn't been filled yet and you should stop at 0 when reading it out; this flag would be cleared the first time that head reaches the end of the buffer and wraps around.

Since you're getting data that are ints, storing them as (larger) strings would be silly; just create an array of ints of the smallest type that can store all possible values of your sensor, e.g., uint16_t if you're getting unsigned 16-bit values. With some experimentation and checking, you should be able to figure out how large a buffer you can use based on how much storage you have free after the RAM for other parts of the program has been allocated. I'd imagine that if you're careful and you're making sure you store strings in program memory you could make room for a buffer that would hold a couple of hundred entries.

Since this seems to be a student assignment, I won't provide sample code here; writing your ring buffer code will prove an excellent exercise in the details of programming.

Improve this answer
2
  • Thanks ! I already heard about ring buffers in Java, this shouldn't be too hard to implement ; but I'm not sure if it'll be enough. Well, can't know without trying, if it's enough I'll mark the answer as accepted then.
    – CLOVIS
    Apr 17, 2017 at 15:38
  • 1
    What do you mean by "enough"? The problem with your question is that it doesn't have any clear criteria, such as the amount of data that must be stored. (If you must store more data than will fit in memory, clearly both this solution and the "Strings in memory" solution you originally used are both completely unacceptable and you should have asked two, separate, different questions: one specifically about how to get the SD Card working with WiFi (detailing your hardware precisely), and the other about other options for external storage.
    – cjs
    Apr 18, 2017 at 0:23
0

What would be the best (or at least a working one) way of achieving this ?

none. Because the upper end of your possibilities (ie. infinite amount of data) is way beyond anything a computer can possibly handle now and into the distant future.

So unless you have some reasonable upper bound for the data in question, there isn't a whole lot you can do to satisfy your application.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.