For the most part, none of the AVR based arduinos are any better at number crunching than any others. They all have a single AVR core running at 16Mhz. None have an FPU, but the software floating point code is very efficient and at the time scales you care about for a calculator, it really doesn't make much of a difference.
The biggest differences between models are memory and available IO, so the decision between AVR models should probably come from what kind of peripherals you want to plug in, not what you want to calculate.
That said, if you want to do a lot of high precision, numerical calculations (like numerical integration or something), you will probably want something with more memory, and floating point types larger than 32-bits. I don't have much experience with those models, but I'm pretty sure that a Yun or a Zero would be better suited for those kinds of calculations. They don't use AVR based main CPUs.
For IO there are too many options to enumerate here. Take a look at the usual suspect's pages (sparkfun, adafruit, etc...) for displays and buttons. These sites have good descriptions and example code for everything.
When picking devices, remember that you can have multiple I2C devices plugged in at once, but only one UART serial per port (uno/leonardo 1 port, mega 4). Software serial can give you more UARTs for low speed comms if need be.
If you want a lot of different individual buttons, there are ways to get more than 1 button per pin, but it would be a bit easier electrically to pick an arduino with as many pins as buttons. Some of the keypads available already have a special wiring scheme to get higher than 1-to-1 pin ratios and descriptions of how to use them, so they might be a good pick to get yourself started.
