It will be always like this when you live in a city. So many people in a limited area. Live and let live, like you said, is a good idea and when it's too much (like very loud music in the night) you can always ask to turn it down a bit.
If you (I speak in general) cannot stand it, go live in the countryside, like the Morvan in France:) It's so quiet there at night, the silence presses on your ears. It's very strange to experience this phenomenon.