You can embed a bootloaded ATMega328 (or any of the other compatible ATMega processors) http://arduino.cc/en/pmwiki.php?n=Main/Standalone in your project. That way, you will only need the processor and a couple of components, setting you back only a couple of dollars and saving quite a lot of space:
http://arduino.cc/en/pmwiki.php?n=Main/Standalone
If that is one bridge to far, you can have a look at the Arduino Pro or Arduino Pro Mini. Those boards are complete boards, only smaller and cheaper, and specifically designed for a more definitive use in projects.
If you need even more space and cost reduction, you can switch to a stand-alone ATTiny processor. The better models of those can also contain an Arduino program. Instructables has this great tutorial that shows you how to use an ATTiny85 or ATTiny45 as a lightweight 'Arduino'.
Recently, I have tried to use an ATTiny13A for a project, but that one only has 1KB of flash memory, which leaves you with hardly any space for the Arduino bootloader, let alone your program, so therefor I chose to program it directly using AVR Studio, at which point I think you cannot in any way call it an Arduino project anymore. :) Still, it's not that different. You only miss out on some of the library functions, and the implied application loop.