Holy men don't just advocate about all people. They advocate for their own group, own religion and at the core of it all is for them to grow in numbers. Also, let us not kid ourselves. "Holy men' that supposedly deny their sexuality do enjoy it, just behind close doors. Also they are often homosexuals (as it has been demonstrated many times). Even he denial of sex, and thus turning into a holy man, comes down to be about sex.

I believe at least that all healthy organisms have a healthy sexual function. This is why and how they come to exist. If one doesn't "have it" then they are most likely sick. If it is a choice (egg/chicken thing) then again, denial of sex makes one choices revolve around it. (e.g betterment of humanity).

Also, even if your speculation is correct the outliers you mention are to be expected in any species. Nothing is ever 100% is nature. That's the whole point.