limit length of the generated timestamp from ESP32 with DS3231 to 13

Question

I am using ESP-WROOM-32 (Olimex ESP32-PoE) with DS3231. I have used the a patch from PR on RTClib to avoid the rollover from the DS3231.

Since the DS3231 RTC provides Unix Epoch Timestamps which have a precision of only seconds, I wish to concatenate the millis() to the above mentioned timestamps to provide millisecond precision.

This is critical since I want to store information to InfluxDB from the nodes with ms precision. For that the timestamps should always be 13 digits.

code

#include <RTClib.h>
#include <Wire.h>

// PoE setting for Olimex PoE board

#define ETH_CLK_MODE ETH_CLOCK_GPIO17_OUT
#define ETH_PHY_POWER 12

// Pins for I2C on Olimex PoE board
#define I2C_SDA 13
#define I2C_SCL 16


RTC_DS3231 rtc;
RTC_Millis software_time;

void setup() {
  // put your setup code here, to run once:
  Wire.begin(I2C_SDA, I2C_SCL, 400000);
  Serial.begin(9600);
  if (!rtc.begin()) {
    Serial.println("No RTC Detected");
  }
  // adjust RTC Time
  rtc.adjust(DateTime(F(__DATE__), F(__TIME__)));
  // adjust software timer
  software_time.adjust(rtc.now());

}

void loop() {
  // put your main code here, to run repeatedly:
  DateTime rtc_now_time = rtc.now();
  DateTime software_now_time = software_time.now();
  Serial.println();
  Serial.print("RTC now: "); Serial.print(rtc_now_time.unixtime());
  Serial.println();
  Serial.print("millis: "); Serial.print(millis());
  delay(100);

}

Output

RTC now: 1538502183
millis: 28
RTC now: 1538502183
millis: 128
RTC now: 1538502183
millis: 228
RTC now: 1538502183
millis: 328
RTC now: 1538502183
millis: 428
RTC now: 1538502183
millis: 528
RTC now: 1538502183
millis: 628
RTC now: 1538502183
millis: 728
RTC now: 1538502183
millis: 828
RTC now: 1538502183
millis: 928
RTC now: 1538502184
millis: 1028

I will store the timestamps in String so I was thinking of concatenating the .unixtime() with millis(). However since I want all my timestamps to be of fixed lenght (13) with the increasing millis value seems to be difficult because as soon as the number goes more that 999 the a digit increases making the string length 14.

Side Effect

if the timestamps for ms do not match length of 13, InfluxDB rejects the data passed through

Epoch Time

the output provided above is of 10 digits which is the epoch time since 1970. The precision of the timestamps above is of seconds. If one wants to add milliseconds to it one needs to add 3 more digits behind it. If precision needs to be microseconds it should have 6 digits and so on.. One can check the timestamps above on [https://epochconverter.com]

Any suggestions on overcoming this issue ?

Answer 1

For getting millisecond resolution out of an RTC, I think the only viable solution is to sync with the 1 Hz square wave output, as suggested by Jot in a comment. Note that, since you are only counting seconds and milliseconds, you probably don't care about hours, minutes, days, months, etc. So you won't use the RTC for timekeeping: only for providing the 1 Hz reference signal and, maybe, an initial timestamp at program startup.

Connect the SQW pin of the DS3231 to an interrupt-capable input of your MCU. Add the following to your setup():

pinMode(SQW_PIN, INPUT_PULLUP);
rtc.writeSqwPinMode(DS3231_SquareWave1Hz);
attachInterrupt(digitalPinToInterrupt(SQW_PIN), tick, FALLING);

Use the interrupt routine to both keep track of the total second count (seconds elapsed since the Unix epoch) and record the micros() value of the last tick:

volatile uint32_t seconds_since_epoch;
volatile uint32_t micros_on_tick;

void tick() {
    seconds_since_epoch++;
    micros_on_tick = micros();
}

Now you can combine these two values to get a millisecond-resolved timestamp as follows:

uint64_t millis_since_epoch() {
    noInterrupts();
    uint32_t seconds = seconds_since_epoch;
    uint32_t microseconds = micros_on_tick;
    interrupts();
    uint32_t extra_micros = micros() - microseconds;
    uint16_t extra_millis = extra_micros / 1000;
    if (extra_millis > 999)
        extra_millis = 999;
    return (uint64_t) seconds * 1000 + extra_millis;
}

The value returned from this function is the 13-digit number you want, provided seconds_since_epoch has been sensibly initialized.

Answer 2

the goal is to generate unique and ordered millis part of timestamps in one epoch second.

unsigned long thisNowMillis;
uint32_t thisNow;

-

 uint32_t now = rtc_now_time.unixtime();
 if (now != thisNow) {
   thisNow = now;
   thisNowMillis = millis();
 }
 Serial.print("millis: "); Serial.print(millis() - thisNowMillis);

not tested and it will not work for counting intervals between timestamps.