Or flagging / downvote should cost like on StackOverflow. If you downvote on StackOverflow both the person downvoting and the person being downvotes loose reputation.

As a result you only downvote if the post is so bad that your are prepared to pay the price of the downvote. It still happens because some postings are that bad.