Having noise in the sensor data is common and normal, especially on low end sensor.
As you can imagine there is a "data science" to clean up the values, mostly based on statistic.
One of the easiest trick is to remove all values that have a difference > Xt from previous value; as you can imagine the tolerance Xt is calculated by a factor X, let's say a bit more the maximum speed you think the system will operate in time multiplies by the time since the last valid measure.
For example you expect a max distance chance of 10m in one second, so we use as tolerance 15m/s. If we don't read valid value, after 6 seconds our tolerance is 15m/s * 6s = 90m
Of course after much time maybe you just want to accept the first reading that seems valid, so you analyze 10, 100, 1000 value and see if they seems correct.
Then for a quick and dirty smoothing and cleaning if the remaining data, you can run a average on the value, there are different forms, and more value you use more stable are your output, but also less fast to follow changes.
A kallmann filter is the most common but also complex way to clean the data, but I guess in most hobbyist application is just overkill.