When you reset a Uno running the Optiboot loader, the bootloader first flashes pin 13 three times.

Top line (gray) is sent to the Arduino, middle line (orange) is sent from the Arduino.
While that is happening, the program avrdude
running on your computer is sending a query to the device:
STK_GET_SYNC / CRC_EOP (0x30/0x20)
The Arduino doesn't notice the first "get sync" because it is busy flashing pin 13. Once it is done it notices the "get sync" (it would be buffered by the serial hardware) and replies:
STK_INSYNC / STK_OK (0x14/0x10)
It looks like avrdude got a bit impatient, and timed out, because it tries again with the "get sync" query. This time Optiboot responds immediately.
The rest of the uploading is described in the next image. Example produced uploading the stock "Blink" program.

(Click on the image above for a larger version)
The steps are:
- Query: Get Sync? Reply: In Sync.
- Query: Get parameter? (major version) Reply: version 4.
- Query: Get parameter? (minor version) Reply: version 4.
Set device parameters. The following device parameters are sent to the chip:
0x42 // STK_SET_DEVICE
0x86 // device code
0x00 // revision
0x00 // progtype: “0” – Both Parallel/High-voltage and Serial mode
0x01 // parmode: “1” – Full parallel interface
0x01 // polling: “1” – Polling may be used
0x01 // selftimed: “1” – Self timed
0x01 // lockbytes: Number of Lock bytes.
0x03 // fusebytes: Number of Fuse bytes
0xFF // flashpollval1
0xFF // flashpollval2
0xFF // eeprompollval1
0xFF // eeprompollval2
0x00 // pagesizehigh
0x80 // pagesizelow
0x04 // eepromsizehigh
0x00 // eepromsizelow
0x00 // flashsize4
0x00 // flashsize3
0x80 // flashsize2
0x00 // flashsize1
0x20 // Sync_CRC_EOP
Optiboot ignores all those and replies with In Sync/OK. :)
Set extended device parameters:
0x45 // STK_SET_DEVICE_EXT
0x05 // commandsize: how many bytes follow
0x04 // eeprompagesize: EEPROM page size in bytes.
0xD7 // signalpagel:
0xC2 // signalbs2:
0x00 // ResetDisable: Defines whether a part has RSTDSBL Fuse
0x20 // Sync_CRC_EOP
Optiboot ignores all those as well and replies with In Sync/OK.
Enter program mode. Reply: In Sync/OK.
Read signature. Optiboot replies with 0x1E 0x95 0x0F
without actually reading the signature.
Write fuses (four times). Optiboot does not write the fuse but just replies In Sync/OK.
Load address (initially 0x0000). The address is in words (ie. a word is two bytes). This sets the address for where the next page of data will be written.
Program page (up to 128 bytes are sent). Optiboot replies "In Sync" immediately. Then there is a pause of about 4 ms while it actually programs the page. Then it replies "OK".
Load address (now 0x0040). This is address 64 in decimal, ie. 128 bytes from the start of program memory.
Another page is written. This sequence continues until all the pages are written.
Load address (back to 0x0000). This is for verifying the write.
Read page (up to 128 bytes are read). This is for verifying. Note that even if the verify fails, the bad data has already been written to the chip.
Leave programming mode.
What does "not in sync" mean?
As you can see from the above, every step through the programming sequence the Arduino is expected to reply with "In Sync" (0x14), possibly followed by some data, followed by "OK" (0x10).
If it is "not in sync" that means that avrdude did not get the "in sync" response. Possible reasons could be:
- Wrong baud rate used
- Wrong serial port selected in IDE
- Wrong board type selected in IDE
- No bootloader installed
- Wrong bootloader installed
- Board not configured to use the bootloader (in the fuses)
- Some device plugged into pins D0 and D1 on the Arduino, interfering with serial commications
- The USB interface chip (ATmega16U2) not working properly
- Wrong clock for the board
- Wrong fuse settings on the Atmega328P (eg. "divide clock by 8")
- Board/chip damaged
- Faulty USB cable (some USB cables provide power only, and are not for data, eg. cheap cables for USB fans)
What is "in sync"?
As mentioned above, the response "In sync" means that the Arduino (bootloader) is synchronised with the uploading program.
What protocol is being used?
The protocol is the STK500 protocol as documented by Atmel. See the references below.
References
Note: STK500 Version 2 is not used in Optiboot, but it is included for information in case you are using boards like the Mega2560.
STK500 constants
/* STK500 constants list, from AVRDUDE */
#define STK_OK 0x10
#define STK_FAILED 0x11 // Not used
#define STK_UNKNOWN 0x12 // Not used
#define STK_NODEVICE 0x13 // Not used
#define STK_INSYNC 0x14 // ' '
#define STK_NOSYNC 0x15 // Not used
#define ADC_CHANNEL_ERROR 0x16 // Not used
#define ADC_MEASURE_OK 0x17 // Not used
#define PWM_CHANNEL_ERROR 0x18 // Not used
#define PWM_ADJUST_OK 0x19 // Not used
#define CRC_EOP 0x20 // 'SPACE'
#define STK_GET_SYNC 0x30 // '0'
#define STK_GET_SIGN_ON 0x31 // '1'
#define STK_SET_PARAMETER 0x40 // '@'
#define STK_GET_PARAMETER 0x41 // 'A'
#define STK_SET_DEVICE 0x42 // 'B'
#define STK_SET_DEVICE_EXT 0x45 // 'E'
#define STK_ENTER_PROGMODE 0x50 // 'P'
#define STK_LEAVE_PROGMODE 0x51 // 'Q'
#define STK_CHIP_ERASE 0x52 // 'R'
#define STK_CHECK_AUTOINC 0x53 // 'S'
#define STK_LOAD_ADDRESS 0x55 // 'U'
#define STK_UNIVERSAL 0x56 // 'V'
#define STK_PROG_FLASH 0x60 // '`'
#define STK_PROG_DATA 0x61 // 'a'
#define STK_PROG_FUSE 0x62 // 'b'
#define STK_PROG_LOCK 0x63 // 'c'
#define STK_PROG_PAGE 0x64 // 'd'
#define STK_PROG_FUSE_EXT 0x65 // 'e'
#define STK_READ_FLASH 0x70 // 'p'
#define STK_READ_DATA 0x71 // 'q'
#define STK_READ_FUSE 0x72 // 'r'
#define STK_READ_LOCK 0x73 // 's'
#define STK_READ_PAGE 0x74 // 't'
#define STK_READ_SIGN 0x75 // 'u'
#define STK_READ_OSCCAL 0x76 // 'v'
#define STK_READ_FUSE_EXT 0x77 // 'w'
#define STK_READ_OSCCAL_EXT 0x78 // 'x'