Traditionally, I lean more towards "no" than "yes", so I definitely know where you're coming from. I did an experiment a few years ago where I said "yes" to everything, kind of like that movie Yes Man just so I could move far away from my comfort zone. The experiment had mixed results, but it did give me a new perspective.
I fully agree with all your sentiments above, especially with your second point, as I feel it resonates with me. Time is a limited resource that we can't afford to spare most of the time, so managing how we use it is key to doing more, helping more.
It's great to look at both sides of the equation, instead of just doing one or the other. That way, we wouldn't have any reservations with regard to choosing which path to take when we come to it.
This is such a great read! No wonder you have votes numbering thousands!