how long will that value "stick" to that address before being lost?
The Atmega328 data retention is guaranteed by Atmel (the manufacturer) for 20 years at 85 degrees Celsius, 100 years at 25 degrees Celsius. Beware cheap Chinese clones usually use counterfeits chips, meaning those values are not going to be guaranteed any more
If I were to power down the Arduino, flash a whole new program to it, and power it on, could I still retrieve that old/cached value written from the previous program?
While @ignacio answer is more detailed I believe you are interested in plain Arduino IDE programming along with standard Arduino boards, in which case the answer is yes, the data is not going to be erased.