I'm so sick of my stutter. Why is it okay when I am confident and horrible the moment I lose my confidence?

I'm so sick of my stutter. Why is it okay when I am confident and horrible the moment I lose my confidence? I thought for a moment I overcame my stutter as I was talking to a very supportive colleague yesterday over dinner, and I was speaking quickly and fluently, and for some reason I thought the stutter went away, or that it was all in my mind. I did stutter with him also, but not that much. Note that I've been experimenting speaking slower, however, I speak at my natural fast pace with my colleague because I am quite close to him. Fast forward to today, where I felt fluent because of the great interaction I had yesterday and introduced myself to a new co-worker. Once again, like in so many other situations, we didn't click because I was incredibly held back due to my stutter, being quiet, and when I did talk, I stuttered a lot, and made awkward conversation, as like 70% of the things I wanted to say, I felt were stuck in my throat and I had to abruptly end sentences/change what I would say to not stutter or block. He made some excuse to end the conversation and went to talk to someone else and pretended that we never met after that. Then after I became very unconfident the rest of the day. This is when my "real" stutter came back. Went to ask a receptionist a few hours later where a certain shop was, and stuttered **a lot more** through the interaction, except it was worse than it was at the start of the day. Stuttered more with other people I spoke with throughout the day as well. This new amplified stutter stayed with me for the entire rest of the day until I got back home and saw family. **Anyone else experience this?** Confident one time of day, and then one bad comment/interaction, and your stutter becomes 10x worse. And obviously because of this, people treat you even worse. It is a horrible feedback cycle. I haven't met anyone whose speech gets as messed up during the same day.