Re-visiting an old question... as I found a very informative blog post
that sheds new light into it. But let me first provide some context
before giving the link.
When assessing the quality of a time base, be it a crystal, a ceramic
resonator or a lab-grade frequency standard, there are two notions that
should be distinguished:
- accuracy: how close is the frequency of the time base to its
nominal value
- stability: how much does that frequency drift over time
Accuracy is important if you want your clock to give correct time “out
of the box”. However, if you are willing to spend some time calibrating
your clock, then you do not really care because you are going to
calibrate out any inaccuracy you measure. jfpoilpret's answer provides
an example of a “manual” calibration protocol, which is by necessity
quite lengthy. If you can borrow a GPS module with a 1PPS output, the
calibration could be done in a few seconds.
Stability is a more serious issue. If the frequency of the time base
drifts randomly, this will defeat your calibration efforts. Essentially,
the calibration will tell you how fast or slow your clock is running
right now, but it will not allow you to predict how fast or slow it
will run in the future.
Here is the promised link: Arduino clock frequency
accuracy, by Joris van Rantwijk.
What Joris did is measure the accuracy and stability of an Arduino Pro
Mini (clocked off a ceramic resonator) and an old Duemilianove (quartz
crystal). From my perspective, the main takeaways are:
- both clocks are grossly inaccurate, thus both would need user
calibration in order to be used as timepieces
- the quartz crystal of the Duemilianove has decent stability, better
than 1.5e-8 at 6 h averaging time
- the stability of the Pro Mini’s ceramic resonator is pathetic, more
than two orders of magnitude worse than the crystal, which makes it
essentially useless as a time piece
Here is his Allan deviation plot, which measures clock instability
as a function of observation time:

(source: jorisvr.nl)
Although this study has some limitations (only two boards were tested,
and the observation time is too short), it is well thought and very
informative. I encourage you to read it in whole.