There are two issues at play here:
1 – Limited precision of the float data type
A float has a precision of roughly 10−7, and any number that
is not an exact float is supposed to be rounded to the nearest float.
For example, the two floats closest to 1.234567 are
10356298 × 2−23 = 1.2345669269561767578125 and
10356299 × 2−23 = 1.23456704616546630859375.
The latter is the closest.
2 – Naive string to float implementation in the Arduino core
The Arduino core aims at being small and consume few resources.
Sometimes it cuts corners, at the cost of accuracy. In this case,
although a correct implementation is supposed to round 1.234567 up,
because the nearest float is above that number, the actual
implementation ends up rounding down. Thus you end up with roughly
1.234566927, which is off by 0.61 ULPs (units of the last place), instead
of the expected error of 0.39 ULPs.