Not at all.
It's not jealousy because every user is sharing the daily reward pool so it's in our interest to divvy up the pot as we see fit. For most users their downvote doesn't even affect the payout of a post appreciably, but if they feel a post is getting more than it deserves, then there should be a way of reducing it.
Also, there are posts where someone is posting something harmful to society at large. Once again it's a pool of rewards and thus we have a say in how our reward pool should be shared and with whatever influence you have earned by your SPower you should be able to wield it.
We can only hope that people will use their influence judiciously and not out of spite, envy or some other abusive way. Flagging is more than just for "abuse", just most users here don't know that. Sadly there still isn't a best practices faq or something of that sort for everyone to read that can really explain how things like this work. There is a wiki though that is very informative. Steemwiki.com
it's one thing to flag because something is harmful within the post. Although "harmful to society" is often in the eye of the beholder and whose society.. the beholder or the writer. The difference could be cultural.
It is still quite another to flag simply because the writer is getting more than that person thinks they should. Obviously other readers have already shared as they see fit, who is anyone else to decide they want to take some of that away? Still comes back to jealousy or envy.