The ATMega328p is not rated to operate at 16 MHz on a 3.3v supply.
In practice it may sometimes work, but cannot be considered reliable.
The data sheet will show the actual clock rate vs. voltage tradeoffs, so there might be an intermediate value that is workable for you.
The Arduino toolchain needs to know the CPU clock frequency for the serial baud rates, timing delay code, and any time-based use of the timer peripherals to come out right.
There is a special case where if the configured clock rate and the actual clock rate differ by a factor of two, you may be able to cover that by selecting a similarly wrong baud rate - however, the baud rate in the bootloader was set in comparison to the expected clock frequency when it was compiled before flashing, and the baud rate expected of the bootloader is set in the boards file, so less readily changed than it is in something like the serial monitor.