Ironically, I generally don't go in for the nemesis. "Returning antagonists," absolutely – characters who have different interests and different intents than the protagonists of the players at the table, but without those you don't have much of a story anyway. Absent conflict, it's a travelogue.

Every interaction with another character has the potential to create someone that the people at the table want to see come back, either because business is not completed with that character or because they are legitimately interested in finding out what happens to the story of that character rolling forward. Every time you introduce a character, your goal is to be creating something that people want to continue to engage with.

Sometimes the character is just background. There is no shame in that. In fact, it's perfectly reasonable for the world to be populated by people who aren't important to the story. But you should always have an eye to something about every character which could attract attention. Is it an accent? Is it a disability? Is it a fragment of back story that they drop at random? Is it the promise of some relationship with the character that already is known to the players? Something should connect or at least have the potential of connecting.

When it does, you end up with an interesting story about interesting characters that people care about. When it doesn't – "it's just one damned thing after another."