As far as I understand it checks for a valid serial connection (new program incoming) and then either writes it to flash or starts executing the existing program.
Can't the 32u4 or EDBG fill the role of an ISP and program the MCU instead of the MCU self-programming? I know the EDBG is accessed via OpenOCD to write the bootloader to the SAMD21 on Arduino Zero boards so why is the bootloader even required?
The bootloader is there for convenience.
Yes, you understand what the bootloader does and how it operates. However, you have missed out one benefit, and that is evident when you're not using the bootloader. That is, when you're not using the bootloader the same connection can be used for a serial connection to the PC. If the 32U4 were directly programming the main MCU then it would be talking to the SPI pins not the UART pins, so you would need a second connection, or both a connection to the UART and the SPI pins, and much more complex firmware on the 32U4, in order to get a serial connection to the PC.
So the simplest option is just to have a dumb pass-through interface from the USB to the UART pins and have the main MCU self program with a bootloader.
It also means that you don't need any special programming hardware when making your own board - just something (another arduino maybe) to get the bootloader on there in the first place (or buy the MCU pre-programmed) and then use any TTL serial interface to program it with your main program.
