I only stutter with people I'm close with

I only stutter with people I'm close with Well, I've been struggling with my stutter for most of my life (or at least as long as I can remember). It's never been that big of a deal for me though mostly because I used to only stutter with my parents, and when I was younger both my parents worked full-time so I didn't see them all that much except for the weekends. Things got worse for me after I first started dating my wife though. We started going out when we were sixteen and back then it took me a few months until I first noticed the signs that I was beginning to stutter when talking to her. She had no idea I had a stutter at first, so I told her as soon as I noticed the "signs." She didn't take it seriously at first, but she eventually came around and started to realize I had a real problem. That was ten years ago. Can you imagine how worse it has gotten since? Nowadays there are times when having a simple conversation is such a struggle to me that I literally get frustrated and tell her I don't want to talk anymore for a while. I know it may sound childish, but the amount of effort my body has to put into talking (and usually about the stupidest subjects in the world) has me tired and angry. But a couple years ago I realized this: I only stutter with people I'm *really* close with. None of my friends know that I have a stutter. The only people in the world who know are my parents, my wife, and my therapist. But nonetheless I still struggle every now and then when talking on the phone (which I hate so much) or talking to strangers. As for the latter I've noticed I tend to stutter when I get confused about a subject, which makes me less confident, or when I'm asking for directions/help and feel like I'm disturbing the person I'm talking to. It usually makes me very insecure to think the stranger I'm talking to might not want to be talking to me, which then makes me stutter. One thing that proves my theory is that last year I had a manager who reminded me so much of my mother (not physically, but the way she talked, the mannerisms, etc.) and who for one reason or another developed a sort of "motherly" feelings towards me as well. We got along great and in no-time she even started calling me "her work son." We had a very awesome dynamic, and sooner than I realized, I was starting to stutter when talking to her as well... I felt so close to her on a sentimental level that my stutter must've mistaken her for my real mother and it began to come out when I was around her. I left that job before it got worse (for other reasons lol) but it really got me thinking about how weird that is... Other than that I feel I have it pretty lucky, I've met people who have what I call a "full-time stutter," while as for me it's only with certain people and in certain situations. But don't get me wrong though, it gets pretty bad and it's been affecting my relationship in such a way that I sought the help of a therapist since my wife and I have been at odds for quite a long time. Can you imagine being in a relationship with someone who 90% of the time storms out of the room mad that he can't talk and refuses to speak for hours? I feel so bad for her. I know she understands but it's still frustrating. For both of us. Anyway sorry for the long read, I'm just trying to put it out there to see if I can find someone who goes through a similar problem.