This is a very valid concern I was going to voice as well, since It's not that simple, no victim or no potential victim. You cannot drive drunk because there is considerable danger to others, any drunk will tell you this.
I don't agree with bans though, or limitations. Speed limits don't work, traffic lights don't work, gun bans have one predominant purpose, to arm only criminals, and punishing people doesn't either. What the law should be interested in is restitution or making whole, not revenge or retribution. It should be for the victim to decide if revenge or restitution is what they want if anything, once the requirements have been meet about sufficient proof.