In another words, is it possible to develop a custom bootloader so sketches can be uploaded via serial connection made using the RX/TX pins and not via the USB connector (I am using Pro Micro boards)? Thanks!
Yes. You can develop a bootloader for the ATmega32u4 that will receive firmware over any interface (UART, SPI, I2C etc.).
Loading via UART is a solved problem. Look for the AVR109 bootloader.
No need to have a specific bootloader. The basic bootloader can do that (in Arduino that's the case so I hope this answer will apply in your case. But if not, the principle is certainly the same).
Once you perform a reset on the board, the bootloader look at the serial port during a short time. If something come, it checks if it's an hex file and if yes it send it to flash. The USB just use the RX input.
So you can send data using a direct connection or using for example a Bluetooh like HC-05. I do that for some project in order to update via Android: the Android app send an order to the Arduino, through Bluetooth. Receiving this, the Arduino RESET itself, and then the Android App send immediatly data.
On Arduino side: connect for example A0 to RESET PIN and perform a digitalWrite(PIN_RESET,HIGH); Have also a look at Reset an Arduino Uno in code
After the RESET, the board will reply with 0x14 0x10 (so it send this to TX and so if you have a bluetooh module, you'll received than on your Android App (eg))
You must then send 0x50 0x20 to the board to tell it you enter programming mode. Then send 0x75 0x20 to get the id of the board in order to be sure you will send the HEX file for the right hardware. You will receive 0x14 XXX 0x10 where XXX = id of the board
Then you can send the HEX file.
For that, you will send it by blocks of 128 bytes
So start by dividing the HEX to know the number of blocks then loop against this number: Start at ADR=0 (adresse dest in flash) and send:
For each block:
1) Send 0x55, low byte of ADR, high byte of ADR, 0x20
2) check reply (must be 0x14 0x10)
3) Send the 128 bytes: 0x64, 0x00, 0x80, 0x46 then the 128 bytes and finish by 0x20 (0x80 is the size)
4) check reply (must be 0x14 0x10)
5) update ADR. Take care: ADR is a word pointer (2 bytes). So ADR = ADR+0X40
Loop on num of blocks
Finish the job by sending 0x51 0x20 (exit programing mode)
0x46 before the 128 bytes tells you want to write in flash. Other codes are available eg for checking, getting data etc..
- It seems mandatory to send each time 128 bytes. So if your last block is smaller, you need to add some '\0' to get a total of 128. I've been unable to get good results trying to change the 0x80 value and send less bytes. :(
It's not very easy and need tests to get a good code but that really fine to be able to have an Arduino the user can update.
Hope this will help.