2

I have tested this problem on 6 micro-SDHC cards of 3 different types. 3 are 32-GB Sandisk, 2 are 16-GB Kingston, and 1 is 16-GB Samsung. These cards are used in data loggers and typically contain hundreds or thousands of 5-MB text files whose names are formatted like FILE1234.TXT.

The problem is that when a logger containing such a card (containing data) is started, the first write takes a long time--from 7 to 30 seconds. Opening the file (the smallest available filename) doesn't take too long. Subsequent writes don't take long. Just the first write.

Now, the plot thickens: this delay can be eliminated by first opening FILE0000.TXT and immediately closing it. Further, once this is done, that disk is never affected by this delay again. This has happened with each of the disks.

Here is some minimal code demonstrating the issue:

#include <SdFat.h>
SdFat sd;
SdFile file;

void setup(){
  Serial.begin(57600);
  sd.begin();
  int m = millis(); // just to time everything

  // critical line that prevents delays
  file.open("FILE0000.TXT", O_WRITE | O_TRUNC | O_CREAT); file.close();

  // file we actually want to write to
  file.open("FILE6005.TXT", O_WRITE | O_TRUNC | O_CREAT);
  Serial.println(millis()-m); // print how long it took to open

  file.println(F("test"));  
  Serial.println(millis()-m); // print how long it took to write
  file.close();
}
void loop(){
}

When the "critical line" in the code is commented out, I get a big delay in the first write to the file. When it's uncommented, there's no delay, and I never get a delay on that disk again even with the line commented. I saw this on all six disks.

One further bit of mystery: at first, this did not work on one of the cards. I checked and found that this card alone had a hidden ".Trash-1000" directory. I removed that directory and tried again, and this fix did work that time.

Does anyone have any idea what's going on here?

3
  • That's really interesting, I'd like to know what exactly is going on too.
    – Chupo_cro
    Dec 13, 2016 at 6:54
  • Sounds like the SD-FAT library is loading the directory structure, File-Allocation tables, building it's 'representation' of the SD cards usage/layout. Do you experience the delay during the FILE0000.TXT opening? The .Trash-1000 subdir likely triggered a full scan of the directory layout as SD-FAT looked for FILExxxx.TXT. Remember you're on a memory constrained device, and the SD-FAT library must keep it's memory usage minimal, so blocks/sectors are repeatedly read and dumped instead of being cached as is done on (much) larger memory systems. This will slow access dramatically.
    – lornix
    Dec 13, 2016 at 9:29
  • You would probably get a huge speed increase by splitting your files into directories, such as 3/6/FILE3672.TXT
    – Majenko
    Dec 13, 2016 at 11:09

2 Answers 2

1

Most file systems exhibit poor performance if you store many files in one directory. 'Normal' desktop computers are perhaps so fast, that you don't notice the performance impact, but accessing the file system from a relatively slow micro-controller, which also have limited means of caching the file system data structures in RAM, you may have to consider for this and either write a smaller number of files or build a tree of directories, so that you only have a limited number of files in each directory.

1
  • 1
    Why to change the file structure if, as stated in the question, the delay can be eliminated with just one line of the code?
    – Chupo_cro
    Dec 13, 2016 at 15:10
1

I have observed this as well, but with bare-metal programming rather than arduino. I use an STM32F7 chip to write to a class 10 SD card using block writes from the SD card standard. I use a custom file system rather than FAT. My first write to the SD card takes roughly 8-10 times longer than all of my other writes. This is each time I remove power and resupply power to the device. This doesn't happen if I reset the processor without removing the power supply. If I only reset the processor (which means I reconfigure and initialize the SD card) this write delay doesn't happen.

The only thing that doesn't happen to the SD card before the first write is the configuration of the data path state machine (DPSM). Could the DPSM being in a state that isn't conducive to writing on start-up be the culprit?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.