Good to learn that you respect vegans as well as all others. But I'd clarify one thing here. Veganism is not just about self-discipline but it's also about respecting the basic rights of survival for all sentient beings. So they don't care as long as the other person's personal choice does not make any significant impact on the lives of others but if someone is encroaching upon the right to life of some one else, then they take it as their moral duty to protest the act of offender.
And there is one more reason to be worried about. As non-smokers are worried about the adverse impact of passive smoking or the second-hand smoking on their and their environment's health, so is the habit of non-vegan (carnists) impacting the health of the environment adversely. So in a way a carnis is not only encroaching upon the right of another being, he/she is also taking away more share of common environment then he/she is entitled to (as we all share this planet and its environment with all others). This can be called "second-hand eating".
So I totally agree with you to not to rob someone of "that" precious moment ...and that moment means trading the whole life from someone! A life being robbed can not be ignored and certainly deserves more attention then a fun moment for some carnist.