Sorry, you are on this thing called earth. All of your thoughts help create the reality that we experience. Which means, everyone is affecting everyone.

So, you do not get to be an individual. You are stuck here with the rest of us.
And so, your (and my) liberty gets curtailed by living in society.
We don't go around killing people.
We don't go around hurting people.
We stay out of each other's houses unless invited.
We all drive on the same side of the road.

Individual liberty has to be balanced with societal responsibility.