0

I'm using the Adafruit SD shield with a Mega board and wrote a simple program to log data. I am outputting a constant stream of data at 115200 baudrate from another Arduino to the Mega. The data output is mostly good, but there are places where some data are skipped. These skipped data usually occurs around a file flush. I've tried playing around with the flush frequency and the soft buffer size, but to no avail. I'm not sure what these hiccups are caused by and would like some help.

void loop(void)
{
  while (Serial1.available())
  {
    softBuffer[i] = Serial1.read();
    i++;
  }

  if (i >= SOFT_BUFFER_SIZE)
  {
    for (x = 0; x < SOFT_BUFFER_SIZE; x++)
    {
      logfile.print(softBuffer[x]);
    }

    logfile.flush();


    i = 0;
  }
}
  • The amount of time an SD card can take to actually commit data is relatively unbounded. If your buffer cannot handle the data delivered for a fair fraction of a second, you may have problems with some cards. How much data is delivered in a second? 115200 baud can potentially bring as much as 11520 bytes, while a Mega 2560 only has 8192 bytes of RAM in total. – Chris Stratton Aug 22 '14 at 15:12
  • 1
    But if you are not up against that limit, you could still be against another based on the architecture of your code. A different scheme would use two buffers, one being filled directly by the serial interrupt (no Arduino Serial class), while the other is flushed to the card, then swap. Or use a non-blocking SD library which can do other things while periodically polling the card to see if it is done. – Chris Stratton Aug 22 '14 at 15:14
  • @ChrisStratton I don't really understand the dual buffer suggestion. With two buffers, aren't they still printing and flushing to the same file? or do you mean doing something like alternating between logfile1.print and logfile2.print? – user2218339 Aug 22 '14 at 19:24
  • I mean continuing to fill one buffer while waiting for the SD card to flush the other, presumably to a single file. Your current code does not appear to be able to continue accept new data while it is waiting for the card to finish flushing old data. But depending on your data rate, you may not have enough memory on board to cover for the longest possible card delay anyway. – Chris Stratton Aug 22 '14 at 20:01
  • @ChrisStratton IIRC the Arduino internal serial buffer is only 16 bytes long, increasing that might be an elegant solution. – geometrikal Aug 22 '14 at 23:23
1

The SD card library works by filling up a 512 byte buffer. Once full, the data is written to disk.

The flush command updates the directory listing with the correct size for the file and also updates the file allocation table. It only needs to be done once, just before you pull the card out.

While flush is doing its thing, the internal serial buffer of the Arduino is filling up with new bytes. The internal buffer is 64 bytes long so if flush takes too long, the internal buffer fills up and you lose characters. You could edit HardwareSerial.cpp in the Arduino program directory to make it say 256 bytes.

Because of this softBuffer is not needed. Simply write straight to the card and let the internal buffer handle the buffering. See https://github.com/arduino/Arduino/blob/master/hardware/arduino/cores/arduino/HardwareSerial.cpp for the Serial code.

unsigned long bytesWritten = 0;
void loop(void)
{
  while (Serial1.available())
  {
    logfile.write(Serial1.read());
    bytesWritten++;
    if(bytesWritten > BYTES_PER_FLUSH) logfile.flush();
  }
}
  • I tried that, but data is still being skipped. I think I might have to try a different SD card library. Is using a baud rate of 115200 testing the limits of the Mega? – user2218339 Aug 23 '14 at 0:04
  • @user2218339 What did you change the buffer in HardwareSerial.cpp to? It is not really the baud rate, but how much data is being transmitted – geometrikal Aug 23 '14 at 4:33
  • Sorry, I must have read too fast, I didn't see the part where you suggested to change the hardware buffer size. I will give it a try today. – user2218339 Aug 25 '14 at 16:04
  • I got a chance to try it out, and that was definitely the solution! Thanks geometrikal! – user2218339 Aug 25 '14 at 18:17
  • @user2218339 glad it worked - i learnt something too! – geometrikal Aug 26 '14 at 2:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.