The AOUT pin of that device is no more than an amplified audio signal. It is AC coupled, and that means that it is a small signal varying around 0v. It is not suitable for feeding directly into an ADC.
The AC coupling capacitor will be being charged up by the DC offset it is there to remove, and the lack of any drain on that capacitor while the ADC is not sampling means it never discharges as it should. By sampling faster you increase the load and thus it can discharge somewhat.
To properly read that AOUT pin you need to add your own DC offset to half the VCC voltage of your Arduino board, which can be done by adding two 10k resistors. One from VCC (5v) to the analogue input, and the other from the analogue input to ground.
The signal you read will then be "idle" at around 2.5v (512 ADC reading) and vary around that as sound is picked up.
Note that you can't just sample once per second and hope: there is no provision for "did sound happen in the past second". Instead you really need to be sampling all the time.
If all you care about is if some sound above a threshold has happened then you would be better using DOUT connected to a digital input or even an interrupt pin, and using the threshold setting trim pot on the board.