Sort:  

the part that I have a problem with is the ability to disproportionately damage someone by flagging.

My opinion is that a downvote is fine..as long as it can not do any damage to the person who wrote the post.
Expressing your displeasure with what someone says is freedom of speak and should be encouraged.
Trashing his rewards is assault.

For example...suppose I make a comment on a post that I do not like..
Then the person who wrote the post flags me.
He's damaging MY reputation because I had the temerity to disagree.
We all know of certain cetaceans who do that.
Flaggin has a chilling effect on speech.

Thank you for your comments. We are wanting to limit the number of changes in each hardfork going forward, so we will not be removing the downvote in this release. I am unable to say anything about future releases, but we are reading community comments when making those decisions. Keep up the discussion!

If you mean reputation that is an issue with the reputation system and the front end's methods of reacting to account reputations.