I know what you mean, from personal experience, you gotta cut down your time with them. If necessary I don't know but maybe its necessary to cut all connections with them. Your number one priority should be to make the most of yourself and if others are hindering you on that journey, they need to be cut out of your life to the extent necessary!