How do you deal with balancing the limited storage of Arduino with the decrease in speed if you just print everything as it gets measured?

Question

I am working on a simple data acquisition system based on an Arduino Uno. Each run would ideally collect up to 10k data points (all doubles). Obviously, the Arduino doesn't have the capacity to hold all that at once. However, as one would expect, printing out the data as its being acquired significantly slows the process down (by about a factor of 7). I have looked around to find if there's a standard way to deal with this issue in the world of Arduinos, but have come up short. I come from a background where memory is never an issue, so I'm not as well versed in this side of things.

Obviously I don't expect perfection, but any insight or sources would be helpful.

Answer 1

This is all about choosing the right tool for the job.

At the moment you're trying to fit 50 clowns into a mini. While there are ways of doing it (by extending the "mini" outwards with extra "room") it's not what you'd normally do.

Instead you'd hire a coach.

If you have a scenario whereby you need to store more data than an Uno can store then you would typically use a different board with a more powerful chip.

There's plenty out there, many that are cheaper than the Uno...

If you desperately do want to use the Uno then:

• Reduce your data size: consider using smaller variables, such as 16 bit integers and fixed point arithmetic instead of floating point values.
• Improve your data transmission: send binary data instead of ASCII data, and increase your baud rate to the highest that can be reliably achieved
• Consider adding external storage, such as an SPI SRAM chip or an SD card.

Answer 2

My solution is to use FRAM, it is much faster then printing and has almost unlimited read and write cycles. It is also non volatile so if power fails the data does not. They can be gotten I2C or SPI your preference. I purchased 32K x 8 devices for well under $6.00 last year from my favorite china supplier. You can treat it as one big ring buffer and print from it while you load data into it.

Answer 3

Don't use String and others dynamic allocated object in your code. There is not any reason for slow down serial output, only inefficient memory management.