That makes sense.
There would still be some level of a correlation you could detect though right?
The sampling bias would mainly cause a problem for being sure if the correlation indicated causation right?
But it could at least be seen - does downvoting correlate to users not participating anymore. There could still be a question as to whether that's a good thing, a bad thing, or if the downvote was likely the cause or not.
Keep in mind... this is all with only the vaguest of understandings of proper controls and testing on what you're describing - fully aware that my thoughts can be full of blindspots :)
You could see such things as those users who receive downvotes being more likely to leave, but there are many forms of sampling bias that could end up giving you that same conclusion mistakenly. It would take a greater degree of rigor to control for such things. Possible to do but takes more work and expertise.
It might be worth doing these kinds of tests anyway, because it can be interesting to examine even while being aware of the limitations.