4

I am trying to make some data logging conserining physiological data coming from a bitalino. My problem is that I would like to collect samples at something like 1 kHz, that is the rate that the bitalino can read the sensors, but I think that the time needed to write this data to the SD card can't keep up with the speed, and here is why i think that:

I put in the script some timers to check what was consuming the time causing to low sample rate, resulting:

Bitalino: 0, SD: 27
Bitalino: 0, SD: 28
Bitalino: 2, SD: 26
Bitalino: 0, SD: 29

Being that the time in miliseconds.

void loop() {
    time = millis();
    readBIT();
    time2 = millis();
    printDataSD();
    time3 = millis();
    printData();
}

void printData() //Send the data acquired to the pc through the serial port
{
  Serial.print("Bitalino: ");
  Serial.print(time2-time); 
  Serial.print(", SD: ");
  Serial.println(time3-time2);
}

void printDataSD() //Send the data acquired to the SD Card
{
    if (myFile) {
    //Bitalino
    myFile.print(_ECG); 
    myFile.print(",");
    myFile.print(_EMG); 
    myFile.print(",");
    myFile.print(_EEG); 
    myFile.print(",");
    myFile.print(_EDA);
    myFile.print(",");
    //Time
    myFile.println(time);
    myFile.flush();

    } else {
    Serial.print("error opening file, ");
    SD.begin(10);
    delay(1000);
  }
}

I would like to know if there is a way to improve my sample rate.

Thank you!

Here is the readBit() like asked:

void readBIT() //Read IMU data
 { 
  _ECG = analogRead(ECG);
  _ECG = ((_ECG/1024) - 0.5)*Vcc*1000/gECG; //mV

 _EMG = analogRead(EMG);
 _EMG = ((_EMG/1024) - 0.5)*Vcc*1000/gEMG; //mV

 _EEG = analogRead(EEG);
 _EEG = ((_EEG/1024) - 0.5)*Vcc*1000000/gEEG; //uV

 _EDA = analogRead(EDA);
 _EDA = (_EDA/1024*Vcc-0.259388)/gEDA; // revolution
 }
  • The serial prints were just for debuging purpose, i intened to remove them before using the device. What do you mean by "write unncecessary data? _EEG is the name of the variable, being printed it is only the value read. How can I know when i need to flush? could i do it every 2,3,4,5 cicles? – arduinoFan May 14 '17 at 19:07
  • 1
    You should consider buffering the sensor data. Basically set up a timer to read the sensor every millisecond and place the data in a buffer. Then you write to the SD card when the buffer's full. Your readBIT() function has to satisfy a few conditions for this to be possible though. We need to see it. – TisteAndii May 14 '17 at 20:48
  • I added the readBit() function, but how can I know that the buffer is full? – arduinoFan May 14 '17 at 21:56
1

The OP has likely solved this problem by now, but I know that others will be in the same position, so I am answering for them.

I routinely record ten channels of biological sensors including one of 16-bit sound at 8 kHz (plus the associated FFT data) to an SD card using arduino-compatible systems, so I doubt very much that the SD card writing per se is the problem.

Throughput to an SD card is a function of several variables.

  • To maximize write throughput you need to write full blocks of data at the correct block size, since partial blocks take the same amount of time as full blocks.
  • You also need to be ready with more data as soon as the SD card is ready to take more, otherwise precious cycles slip away from you.
  • You need to ensure that your writes are non-blocking (don't try to write more than the buffer can receive) so your loop can process and queue up new data while the SD card is doing its thing.
  • You need to read any timing-sensitive sensors using interrupt service routines that write their data either to a circular buffer or to one of a group of two or three buffers that are cycled between being filled and being emptied.

Printing debug statements to the serial port can take many tens of milliseconds in a loop that otherwise may take only a few microseconds. If you write more than the serial port buffer can hold, the call will be blocking and you will be at the mercy of the PC that's reading the serial port. This will always be bad enough to mess up the program you're trying to debug... I've encountered situations where this introduced delays of up to 10 seconds (the PC was misbehaving, but still...).

A quick improvement to printing debug statements is to maintain a loop counter and only print a debug statement every N loops.

The program in the original question uses a naive strategy in the sense that it acquires some amount of data and then immediately tries to write it to the SD card, waiting until the write is complete before trying to acquire new data. This will never work at high data rates for the reasons given above. If you use this approach, you are likely wasting 90% of your program cycles in waiting for the SD write and the Serial port write. If either one takes too long, you will miss the next data acquisition window.

My advice: your first step is to research and find or write a circular buffer library.

Then set a timer to call an interrupt service routine that acquires data and puts it into the circular buffer.

Write a loop routine that checks to see whether the SD card is available for writing, and then takes just enough data out of the circular buffer to fit in a single block write for your SD card (look at the documentation for the card adaptor and the card).

At that point you will know whether your system can keep up with your needs.

It's possible that your SD card writing itself is OK but your Arduino is having trouble keeping up either because it's too slow or because you don't have enough room for a sufficiently large circular buffer. I doubt this because I've recorded and played back audio at 8 KHz on the Uno, but then I don't know what else you need to accomplish in your program.

If you do determine that the Arduino itself is too slow or has insufficient memory, take a look at the Teensy series of boards [https://www.pjrc.com/teensy/index.html]. They are entirely compatible with Arduino libraries and IDE but they run hundreds of times faster than the Uno at a larger data width, plus they have oodles of RAM and a gazillion pins. They're also much smaller than the Arduino form factor and the community that uses them is very knowledgeable.

If it turns out that SD card write throughput is OK but that it requires more buffering than you have on-board, you can add an SPI ram breakout board for more fast buffering space.

Finally, if the amount of data you need to collect is so large and so continuous that you simply can't buffer it long enough to get it written out to SD, or if the SD card truly can't keep up, or if the data is larger than any available SD card, you may consider sending your data off the board in near-real-time. I send 10 hours of nightly data from a Teensy over a USB virtual serial port to the PC at about 3Mbps, and others have done better. Ethernet output is also available and can be at least an order of magnitude greater.

glhf!

0

Why not write a simple program and try to write values like 1, 2, 3 ... 100 to the SD card with a 1 second delay between writes. Then read the values back and send them via serial link to your PC screen. This way you can make sure that you are writing and reading values in the correct way.

Once this works, you can reduce the delay and see what is the minimum delay between writes that the SD card will support.

It might be advisable to also write a program to fill the SD card with 100 values = 0 to clear it before the next test.

  • I played a bit with SD cards and on Teensy 3.5 it was 8ms Open/write/close – Martynas May 14 '17 at 21:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.