Yes, asking can make people uncomfortable. Consider asking all of your customers periodically. That way if becomes part of normal customer care.
For new customers, it's safe to say things like "Thank you for trying me out" or "So glad to have you; how did you hear about me?" are nonthreatening.
Customers that are unhappy, unable to afford, or just found a better fit or deal for them will rarely tell you. People HATE confrontation.
My interfaces were different. I would mail or email people I hadn't heard from in a while with the following:

"I haven't seen you in a while, and hope everything is going well.
If you've chosen to stop using my services, I would appreciate getting your feedback as to why. Don't worry; I'm not going to be persistent!
Not every connection succeeds or is meant to happen. I respect your opinions, and your thoughts may help me improve what I do."

Mailing this as a handwritten or printed note with a postage paid return envelope was far more powerful than email.