The first thing that happens when you apply power to a board or press reset, is the bootloader executes. That waits for a short time to accept a new sketch over the serial port. If nothing arrives it terminates and executes the existing sketch.
On the Uno that delay is only 2 seconds. However on the Leonardo the delay is much longer (around 5-8 seconds) owing to how the serial port is implemented: it has to allow time for the host computer to re-enumerate the USB device after a reset, and that takes a second or two.
If you don't want that delay you will need to remove the bootloader (i.e., ignore it) and program the board through a hardware programmer and the ICSP pins (which will disable the bootloader for you). You lose the convenience of programming through the USB connection though.
Alternatively choose a different board that doesn't have such a delay for the bootloader - one that uses a different programming method.