In a simplified example, I have a microcontroller functioning as a data acquisition device - sensor data is received over UART, and then saved to an SD card.
This received data is in the form of a 12 byte struct, received at 1kHz. The microcontroller receiving the data is an esp32, with an SD card connected over spi. Not an uncommon issue, but if a typical SD card write is triggered everytime new data is received, it will take longer than the gap between samples to write the data.
What is typically considered the best practice for this type of system, presuming the data will always be received faster than it can be written. I have seen examples of circular buffers, but presumably these will eventually fill completely and samples will be missed.
In my particular example, I have the bonus second cpu core in the esp32. Therefore, my thinking is to use 2 alternating buffers. Data will be received on core 1, and placed into the active buffer. When the buffer is full, incoming data will be written to the second buffer, and the first will be written to the SD card by core 2.
Is this a sensible solution to the problem? I am hesitant to implement anything that simply relies on an improvement in SD write speed, as I will likely need to increase the size of the data payload in the future and don't want to run into this issue again later on.