I think you hit the nail on the head here. A few people self-voting doesn't make a big impact. But when all of the discouraged newbies see how profitable self-voting spam is, they won't be motivated to improve their posts and engage with other users. Why work if you don't have to?
My fear is that the Steemit party is going to go from "super hot orgy" to " just a bunch of dudes masturbating" if we don't focus on the social aspect of this platform. We should discourage antisocial behavior, like farming self-votes.