Why do people settle?
People settle because it is more than fast enough. The most common use is just to print some stuff on a terminal for debuggin. 9600 baud is 960 characters per second, or 12 x 80 character lines per second. How fast can you read? :)
If your program is using the serial port for bulk data transfer, you would choose not to settle.
What is the limitation...
The limits on serial are high. Directly you can use 115200 baud in your programs and it will just work. The Arduino terminal will allow a max of 115200, but other programs such as RealTerm would let you run higher.
Hardware serial will run to 1 M baud. If you read around you will see people have used up to 1 M by directly controlling the UART. You might get benefit of high baud rates for uses such as transmitting via a bluetooth chip. If you are using the hardware serial interface to exchange from chip to chip with just a short distance, then 1 M baud is completely feasible. Think of all the SPI and I2C devices that operate just fine at 1 MHz clock rate.
Over larger distances, you will start to have problems with noise when using logic level (plain 0 to 5V) signalling. To use larger distances, you would add a transceiver to provide robust signalling, commonly RS-232 and less commonly RS-485. With RS-232 you could run a mega bit at distances of 10's of feet.
The microprocessor clock speed will be the real limit. With a hardware UART, the processor must load one byte to the UART every 10 bits (for N81). So when you get to 1 M baud it will be a challenge for the 16 MHz processor to keep the UART supplied with data. A new byte will be sent every 160 clock ticks, which is very few lines of code. For a short burst of data, you might achieve that rate. The message is, the processor will run out of speed before the UART is the limit.
Note, this all applies to HardwareSerial, software serial is very different.