According to the datasheet linked to by hcheung in their comment, datasheet, on page 41 it states "Figure 10-2. MCU Start-up, RESET Tied to VCC".
I believe it would be OK to tie the pin directly to Vcc.
I also believe the graphical data presented in "Figure 10-2" could be miss-leading if the reader didn't know what Vcc meant, and also because there is no horizontal axis to specify that they are talking about elapsed time, not voltage change.
Regarding your comment "Thanks, but now I am not sure if this setup will work with Sparkfun's Pocket AVR Programmer.".
Looking at page 40 of the datasheet, a internal pull-up resistor is built into the MCU.
You could leave the reset pin not connected and see if it still works as expected. Perhaps a high impedance DVOM would show Vcc on that pin. Some people say leave it disconnected if there are no issues from EMI. EEV Blog - Is it ok to leave atmega328p reset pin floating?
The data sheet also mentions the power on reset timeout which may allow you to program the MCU with your programmer once it first starts, (assuming you have tied the reset pin to Vcc and you leave the reset pin disconnected on your programmer).
Another option would be to program the MCU, then connect the reset pin to Vcc through a couple of solder pads you can bridge with a blob of solder.
Once the reset pin is tied to Vcc, using a programmer connected to that pin will likely result in the magic smoke being released from the MCU, your programmer, or both.