2

I am creating an accelerometer data logger that writes data to sd card. After some sort of "warm-up" everything works perfectly for at least ~15 minutes (which is what I have tested so far).

However, during the first few seconds there are some pretty long freezes of the File.write() function (180 ms instead of 50us for normal writes!) which I can identify on an oscilloscope (see picture below, signal low after starting high indicates File.write() phases) and which I can also see in the data if I include in them a milis() timestamp.

Since RAM on the Atmega328 is limited, there is also tight limits to the amount of sensor data I can buffer (I am using an interrupt for transfer from the sensor, where I can store the data to a buffer and thus bridge the shorter sd card delays).

enter image description here

I stripped down my code to the absolute bare bones, just writing nonsense data instead of those from the accelerometer. What could be the reason for these freezes? The data rate is the same during the whole run. Isn't the sd card electrically or logically initialized after SD.begin()? It doesn't seem to be a problem with erasing flash memory blocks because it only occurs in the beginning. Even if I restart the application a second time (after staying electrically on all the time) it already shows no freezes anymore. Whereas if I unplug/plug power, the freezes in the beginning reappear.

And most importantly: how can I solve it?

Of course I could write the dummy data before receiving the start button. But how many? Does it depend on sd card type? Does it depend on cpu or spi clock frequency? I want a robust solution that doesn't break when I use another type of sd.

Code:

#include <SPI.h>
#include <SD.h>

const int chipSelect = 10; // chip select of the sd card
const int pushButton = 9; // this is where the start button is (on a pulldown resistor)
const int debugPin = 1; // a line used for displaying a debug bit on the oscilloscope

const int bufferSize = 14;
byte buffer[bufferSize] = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14}; // for the test just dummy values

String fileName = "accellog.raw";

void setup() 
{
  pinMode(pushButton, INPUT);
  pinMode(debugPin, OUTPUT);

  if (!SD.begin(chipSelect))
  {
    // on error stop execution
    for(;;);
  }
}

void loop() 
{
  // wait for the start button to be pressed
  while (digitalRead(pushButton) == HIGH);
  delay(100); // debounce delay
  while (digitalRead(pushButton) == LOW);

  // remove old file on sd card
  if (SD.exists(fileName)) SD.remove(fileName);

  File dataFile = SD.open(fileName, FILE_WRITE); 

  if (dataFile) 
  {
    // Debugging: indicate start of whole data transfer on an oscilloscope
    digitalWrite(debugPin, HIGH);

    int lastMillis = millis();

    for (long i=0; i<10000; i++)
    {
      // wait until millis changes in order to get SD writes at
      // whole millisecond boundaries (some cycles may get lost,
      // however, during lengthy SD writes)
      int tempMillis = millis();
      while (tempMillis == lastMillis) 
      {
        tempMillis = millis();
      }
      lastMillis = tempMillis;

      // Debugging: indicate file access times by sending 0 to the DSO
      digitalWrite(debugPin, LOW);

      dataFile.write(buffer,bufferSize);

      // Debugging: indicate end of file access on DSO
      digitalWrite(debugPin, HIGH);
    }

    // Debugging: indicate end of whole data transfer
    digitalWrite(debugPin, LOW);

    dataFile.close();
  }
  else 
  {
    // on error stop execution
    for (;;);
  }
}
  • try to open file and then close it without any writes – jsotola Jun 9 '18 at 21:50
  • @jsotola: unfortunately that didn't change anything. But I will see how far I get with writing some nonsense data in advance (in a different file). – oliver Jun 9 '18 at 22:35
  • 1
    I can't really reproduce that. On my SD card I get an initial delay (between the first and second write) of about 30 ms, followed by a batch of writes, then a 20 ms delay, then another batch and so on. The pauses are between about 512 bytes being written out which would be the buffer (in the SD library) actually being written to the SD card. My suggestion is to try a different card and see what happens. Maybe one with a fast write class (eg. the sort you use for taking videos). – Nick Gammon Jun 10 '18 at 1:08
  • @NickGammon: yeah, maybe it's the card. I took the cheapest 16 GB cards I could get. :-) I have also been thinking about data loss on the SPI lines as a cause, but the lines on my board are a lot shorter than what you have in a breadboard setting. Anyways, I have a workaround now, at least. I will have to check what happens with cards that have been written full once... – oliver Jun 10 '18 at 7:28
  • 1
    I don't think data loss would really do it. The data either arrives or it doesn't. I took the cheapest 16 GB cards I could get - it looks like you got what you paid for. :) – Nick Gammon Jun 10 '18 at 7:52
0

As a temporary workaround I have come to the following procedure:

I inserted a function that writes a certain amount of bytes into a dummy file, closes it and finally deletes it, all during setup().

This does indeed seem to cure the freezes if (in my scenario) I write at least 6 blocks (512 bytes each, i.e. total 3072 Bytes). Remarkably there were always exactly 6 freezes visible in my oscilloscope plots. I don't even remotely know what the meaning of this magic number 6 could be...

Just to have some additional safety in case I choose a different type of sd card (and this workaround becomes permanently temporary...), I have decided to write 16 kB into the dummy file. This is ~5 times as much as the empirical minimum value (3 kB) and is still pretty fast so that I don't notice any significant startup delay after power-up.

This empiric approach is still unsatisfactory though, and certainly a piece of mediocre engineering. So I won't mark my answer as accepted, to leave the chance for someone else to give a more well-founded answer.

void writeAndEraseDummyFileToEliminateFreeze()
{
  if (SD.exists("dummy")) SD.remove("dummy");
  File dummyFile = SD.open("dummy", FILE_WRITE);
  int n = 16384/recordSize+1; // write 16kB in chunks of recordSize (just because we have that buffer already
  for (int i=0; i<n; i++)
  {
    // buffer values are uninitialized, but who cares, we'll dispose of them anyway
    dummyFile.write(buffer,recordSize);
  }
  dummyFile.close();
  SD.remove("dummy");
}

void setup() 
{
  // ...other things I have been already doing...

  writeAndEraseDummyFileToEliminateFreeze();
}
|improve this answer|||||

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.