Truth there. The fact that you rarely turn away speaks of your strength. Not everyone can act from a position of strength, sometimes in the real world, we bid our time and wait for a sign of weakness or for support from those who are stronger before we stand up.
I have stood up for someone once and while his oppressor bullied and intimidated me for standing up, he stood aside and watched. He did nothing. I came because I thought both of us could do better against the guy, but he left me to battle alone. Even a friend that came with me, stood aside and watched.
I will not pretend to be a hero when I am not. I speak up when I believe that my speaking up will achieve something. If I feel I am not good enough, I call attention to it, I bring those with the power to attack such toxic behaviour.