Long freezes at the beginning of sd card transfer

Question

I am creating an accelerometer data logger that writes data to sd card. After some sort of "warm-up" everything works perfectly for at least ~15 minutes (which is what I have tested so far).

However, during the first few seconds there are some pretty long freezes of the File.write() function (180 ms instead of 50us for normal writes!) which I can identify on an oscilloscope (see picture below, signal low after starting high indicates File.write() phases) and which I can also see in the data if I include in them a milis() timestamp.

Since RAM on the Atmega328 is limited, there is also tight limits to the amount of sensor data I can buffer (I am using an interrupt for transfer from the sensor, where I can store the data to a buffer and thus bridge the shorter sd card delays).

I stripped down my code to the absolute bare bones, just writing nonsense data instead of those from the accelerometer. What could be the reason for these freezes? The data rate is the same during the whole run. Isn't the sd card electrically or logically initialized after SD.begin()? It doesn't seem to be a problem with erasing flash memory blocks because it only occurs in the beginning. Even if I restart the application a second time (after staying electrically on all the time) it already shows no freezes anymore. Whereas if I unplug/plug power, the freezes in the beginning reappear.

And most importantly: how can I solve it?

Of course I could write the dummy data before receiving the start button. But how many? Does it depend on sd card type? Does it depend on cpu or spi clock frequency? I want a robust solution that doesn't break when I use another type of sd.

Code:

#include <SPI.h>
#include <SD.h>

const int chipSelect = 10; // chip select of the sd card
const int pushButton = 9; // this is where the start button is (on a pulldown resistor)
const int debugPin = 1; // a line used for displaying a debug bit on the oscilloscope

const int bufferSize = 14;
byte buffer[bufferSize] = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14}; // for the test just dummy values

String fileName = "accellog.raw";

void setup() 
{
  pinMode(pushButton, INPUT);
  pinMode(debugPin, OUTPUT);

  if (!SD.begin(chipSelect))
  {
    // on error stop execution
    for(;;);
  }
}

void loop() 
{
  // wait for the start button to be pressed
  while (digitalRead(pushButton) == HIGH);
  delay(100); // debounce delay
  while (digitalRead(pushButton) == LOW);

  // remove old file on sd card
  if (SD.exists(fileName)) SD.remove(fileName);

  File dataFile = SD.open(fileName, FILE_WRITE); 

  if (dataFile) 
  {
    // Debugging: indicate start of whole data transfer on an oscilloscope
    digitalWrite(debugPin, HIGH);

    int lastMillis = millis();

    for (long i=0; i<10000; i++)
    {
      // wait until millis changes in order to get SD writes at
      // whole millisecond boundaries (some cycles may get lost,
      // however, during lengthy SD writes)
      int tempMillis = millis();
      while (tempMillis == lastMillis) 
      {
        tempMillis = millis();
      }
      lastMillis = tempMillis;

      // Debugging: indicate file access times by sending 0 to the DSO
      digitalWrite(debugPin, LOW);

      dataFile.write(buffer,bufferSize);

      // Debugging: indicate end of file access on DSO
      digitalWrite(debugPin, HIGH);
    }

    // Debugging: indicate end of whole data transfer
    digitalWrite(debugPin, LOW);

    dataFile.close();
  }
  else 
  {
    // on error stop execution
    for (;;);
  }
}

Answer 1

As a temporary workaround I have come to the following procedure:

I inserted a function that writes a certain amount of bytes into a dummy file, closes it and finally deletes it, all during setup().

This does indeed seem to cure the freezes if (in my scenario) I write at least 6 blocks (512 bytes each, i.e. total 3072 Bytes). Remarkably there were always exactly 6 freezes visible in my oscilloscope plots. I don't even remotely know what the meaning of this magic number 6 could be...

Just to have some additional safety in case I choose a different type of sd card (and this workaround becomes permanently temporary...), I have decided to write 16 kB into the dummy file. This is ~5 times as much as the empirical minimum value (3 kB) and is still pretty fast so that I don't notice any significant startup delay after power-up.

This empiric approach is still unsatisfactory though, and certainly a piece of mediocre engineering. So I won't mark my answer as accepted, to leave the chance for someone else to give a more well-founded answer.

void writeAndEraseDummyFileToEliminateFreeze()
{
  if (SD.exists("dummy")) SD.remove("dummy");
  File dummyFile = SD.open("dummy", FILE_WRITE);
  int n = 16384/recordSize+1; // write 16kB in chunks of recordSize (just because we have that buffer already
  for (int i=0; i<n; i++)
  {
    // buffer values are uninitialized, but who cares, we'll dispose of them anyway
    dummyFile.write(buffer,recordSize);
  }
  dummyFile.close();
  SD.remove("dummy");
}

void setup() 
{
  // ...other things I have been already doing...

  writeAndEraseDummyFileToEliminateFreeze();
}