Could you give us some examples of what you consider unfair downvoting please? While you're at it, possibly some alternatives to downvoting?

HiveVote allows you to put someone on a downvote trail

This I agree is wrong, but then again, I disagree with auto upvotes and trails too.