I absolutely agree with what you are saying and understand the feeling very well.
I think it's important to realize that when it comes to big life decision, despite the fact that fear is supposed to be a deterrent to recklessness, fear is generally unproductive. You need a clear mind and fear clouds it.
Another thing to be aware of is, that we are actually always vulnerable, like it or not. Protecting yourself from vulnerability is not really possible. Life is always a risk and not taking a risk hides the risk of a missed opportunity.
That's why I try to make the best decisions based on the data currently available and my ambitions and desires and when looking back, I only evaluate mistakes in the sense of learning from my experiences and try not to allow regret to play any significant role.