You are defining the issue correctly, it is faithfulness. What is happening is that you are going outside of the relationship to gets your needs met. It could be shopping, working out, or anything to momentarily change your mood, now you are doing it and it is a habit. Along with that you are both rejecting each and betraying one another and this breaks the trust between you. I would suggest you both learn to be honest emotionally about each other painful emotions, and talk about what it feels like. Truly respecting each other means keeping your relationship exclusive toward each other, recognizing when you you need attention or affirmation, and committing to being there for each other.