I am assuming that the digital pins in Arduino are positive (are they 5v?).
On a 5V Arduino, when a GPIO pin is set to OUTPUT and switched HIGH then yes, it is (almost) 5V. When switch LOW it is (almost) 0V or GND. When it is set to INPUT it is neither - it is "high impedance" and senses the voltage on the pin using a set of thresholds.
When the voltage being applied to the pin is above the high threshold (known as VIH) the pin sees it as a logic HIGH and
digitalRead() reports it as such. When the voltage being applied is below the low threshold (known as VIL) the pin sees it as a logic LOW and
digitalRead() reports it as such.
So why can't the potentiometer be connected in the same way?
So yes, you can connect a potentiometer up to to a digital pin in the same way as you can to an analog pin (one pin to +5V, one pin to ground, and the middle "wiper" pin to the digital input):
When you turn it, instead of reading a value between 0 and 1023 as you would on an analog pin, you will either get a HIGH or a LOW depending on how far you have turned the knob and in which direction. In the region in the middle you will read "who knows?" - that is, maybe HIGH, maybe LOW, depending on the weather and the pin itself.
Note that connecting a potentiometer up like this on a pin set to OUTPUT is dangerous (not to you, to the Arduino) and has the potential to damage the GPIO pin's internal circuitry when the pot is turned all the way one way or the other, since it creates a dead short circuit to either ground or +5V, and the GPIO pin itself, when in OUTPUT mode, is also connected internally to either ground or +5V.
Some pins on microcontrollers (not sure about the Arduino off hand) have Schmitt Trigger inputs. These avoide the "who knows?" region by toggling the input HIGH or LOW depending on when the voltage passes a threshold. For the pin to change from reporting a LOW to a HIGH the voltage has to pass above the high threshold. To then switch to reporting a LOW again it has to drop all the way down to the low threshold. Any voltage in between the two thresholds is ignored completely and the previous state is retained.