I have two 6 volts DC motors connected to L298N motor Driver. Do I need to use 12 Volts battery?
No, you need to use a (roughly) 6V battery. They are 6V motors, so they need a 6V battery.
However, you do need to sum the currents not the voltages. The battery must be able to provide at least the current required by both motors together.