I completely understand where you're coming from and understand your concerns, so I really don't blame you at all.
I personally don't think that anyone should be scared to flag if they truly believe that what they're flagging deserves it. I also think that it's important to explain why you flagged, you know?
I think that in the future, flagging may become more automated, but as it stands at the moment, it's kind of our duty as citizens of the Steem planet haha. But yeah, I feel ya on wanting to hold off on the flagging.