It seems in certain applications, for example signal processing, Arduino and other micro-controllers in general run into the overflow error when they're trying to store even moderate amounts of data. It would seem the obvious answer would be to simply take this data, push it onto the computer, and deal with it there instead, where you have much more available computing power. Is this possible with Arduino, or am I missing something that makes it not feasible?
One of the "features" of most microcontrollers/microprocessors is that they don't have a whole heap of RAM - this keeps costs down, and lets the manufacturers keep everything on one chip (unlike your PC).
However to "push" data to somewhere that can process it, you need to have a fast communications channel. SPI is quite fast, but you don't generally talk to PCs using SPI. Serial communication is slower, but if it gets converted into a USB interface (such as on a Uno) you can get respectable speeds talking to a PC. Something like 115200 baud is usually reliable, which lets you transmit 11520 bytes per second.
However it depends on what you want to send (ie. how many bytes), how you are going to indicate where one piece of data ends and the next one starts, maybe allow for error detection or correction, and so on.
Say, for example, it takes 5 bytes to send one piece of data (eg. 4 digits plus a delimiter). Now you are down to 11520/5 = 2304 pieces of data per second.
This may or may not be adequate for whatever-it-is you are trying to do.