No, the frequency of the "buzzer" has no relationship to the voltage.
The duty cycle of the generated tone is proportional to the average voltage though. Typically the duty cycle is 50% - that is it's turned on for half the time, and turned off the other half the time, to make a square wave. This means that the average voltage is 50% of the supply voltage.
If you were to low-pass filter the signal (say through an RC filter as many people do to create a crude DAC) you would end up with a voltage around 2.5V.
Changing the frequency doesn't change the duty cycle, it only changes the audio pitch.
The only caveat to that is that if the buzzer draws more current that the IO pin (or the power supply) can source then it may cause the voltage on that IO pin to droop. That has nothing to do with the frequency though and it would droop just the same at all frequencies. Basically the IO pin has a small resistance to Vcc through the MOSFET that drives it internally - the higher the current the more voltage is dropped across that resistance, and the lower your resultant output voltage. Draw too much current and the heat dissipated by that MOSFET's resistance becomes too great and damages the chip.