It's difficult to have a parent to whom you feel some duty of care who then responds with hurtful, thoughtless behaviour.

Have you considered using the tool you express yourself strongest with to layout to your father just how his behaviour impacts you? Write him a letter. I have in the past found that helpful. It lets me give my words thought and frame them in context that can get lost when a conversation goes sideways. It also lets him read it and give it some thought. Don't be around when he reads it.