Good post.
Down-voting needs to stay. The rule for Down-voting needs to be exactly the same as for up-voting (just in the negative). If you like a post, agree with it, or find the argument or post beneficial in some way then Up-Vote it. However it you dislike a post because the argument is flawed, fined the argument or post non-beneficial then you should be able to down-vote it.
If not then you have differing rules for why to up-vote to those of why to down-vote.
I do feel there needs to be a comment posted with every down-vote for some form of accountability.