He is possibly worth your time. We all make mistakes but if this happens again, I would consider leaving and seeing if he actually changes that. The fact that he knows how much it affects you is what is messed up