SD Card store struct: force big-endian / little-endian?

Question

Following MAJENKO's tutorial on sd card file optimization and data integrity, can I force the SDCard to read and write in either big-endian or little-endian?

I need this in order to maintain backup data integrity across a wide array of IoThings, using various boards (Arduino UNO, MEGA, Nano, NodeMCU, etc) for future parsing.

Answer 1

Many full C libraries have an sys/endian.h header file. This defines a number of macros so you can find out what the endianness of your device is. It also usually defines a number of handy macros that can convert the endianness for you.

htobe16(x)
htobe32(x)
betoh16(x)
betoh32(x)

These convert Host to Big Endian values and back again.

htole16(x)
htole32(x)
letoh16(x)
letoh32(x)

And these convert Host to Little Endian values and back again.

Not many embedded C libraries provide it, though, and not all that do provide the handy macros.

It's not too hard to implement yourself, but it does mean you deciding what to do depending on the target architecture. Which kind of negates the handiness of the endian.h file.

So really you want to re-implement the functionality in a form you can include in all your sketches.

Fortunately the compiler provides a set of macros defining the endianness that you can use though:

#define __ORDER_LITTLE_ENDIAN__ 1234
#define __FLOAT_WORD_ORDER__ __ORDER_LITTLE_ENDIAN__
#define __ORDER_PDP_ENDIAN__ 3412
#define __ORDER_BIG_ENDIAN__ 4321
#define __BYTE_ORDER__ __ORDER_LITTLE_ENDIAN__

So you can examine the __BYTE_ORDER__ macro to see what byte order it equates to and use that to select whether htobe32(x) has to do anything to the byte order or not.

In the PIC32 compiler it's done as a set of macros that either call swap32 / swap16, or pass the value unchanged:

#if __BYTE_ORDER__ == __LITTLE_ENDIAN__

#define htobe16 swap16
#define htobe32 swap32
#define betoh16 swap16
#define betoh32 swap32

#define htole16(x) (x)
#define htole32(x) (x)
#define letoh16(x) (x)
#define letoh32(x) (x)

#endif /* __BYTE_ORDER__ */

#if __BYTE_ORDER__ == __BIG_ENDIAN__

#define htole16 swap16
#define htole32 swap32
#define letoh16 swap16
#define letoh32 swap32

#define htobe16(x) (x)
#define htobe32(x) (x)
#define betoh16(x) (x)
#define betoh32(x) (x)

#endif /* __BYTE_ORDER__ */

So now depending on the byte order it will either swap the bytes around, or leave them unchanged. It's now just a matter of implementing swap16 and swap32 - which I'll leave to you (hint: it just returns the value with the bytes in the reverse order - bit shifting and masking is good for that).

Then you need to decide on an endianness for your file. Maybe choose little-endian since that is probably the most common. So whenever you place a value into a struct for writing you pass it through either htole32(...) or htole16(...) depending on the data type. Then whenever you read from a struct that has been read you pass the value through either letoh32(...) or letoh16(...) to get it in the host's endianness.