Your looking to deep into what others think about you, instead of what you believe to be good.
Are you living to be approved by society or are you living to be good person under what your faith and beliefs are?
As a husband if you argue with your wife doesn't make you bad, it makes you both human.
We all have different views, sometimes you don't look at it as an argument but as a debate over what you each believe or how you feel about something.
People have arguments or debates to show, we have different opinions.
We have differences about a situation , hopefully an outcome that you come to is an agreement in the end, whether good or bad.
You come to a conclusion that you either work it out and bend a little for each other or go separate ways, but that wouldn't make you a bad person.
I'm saying, which in a spiritual sense, we all have a human body.
With that comes humans instincts and humanistic flaws, where as one who walks in the spirit strives to be a good person, resisting the flaws but accepting we will fail at times, the spiritual nature fights with the humanistic side and its a constant battle that we all experience.
Once you let go of the guilt that the human body has desires, and trust in the spiritual side that its natural to see another woman and think she is attractive, the bad is when you act upon those desires or say that out to your wife that the other woman is attractive and make her feel inferior or less attractive.
Accept your not perfect and never will, accept your flaws and strive to improve and respect for others will always prove you to be a good person.