There does not exist the same incentives to abusively downvote as to upvote. It can in some cases be a problem but I think the concern is largely overblown. SP and voting power are valuable scarce resources that can be used to earn money. People who use their votes for malicious downvoting are incurring a cost to themselves (opportunity cost). That isn't enough to ensure it never happens, but perhaps it is just something that will be rare in practice can be considered an acceptable cost (not saying this is necessarily true, but it could be).

I do think there is a bigger problem with this whole idea, which is "good downvoting" being an undersupplied public good. In all likelyhood as the system matures and becomes more competitive we will see very little downvoting, even in cases where it is actually needed.