I can relate to your story so much. I've had a very similar experience with my stutter. One of my past jobs was waiting tables and at the time everyone I knew and even myself was like this is going to be a train wreck. The very same reasons you posted applied to my situation as well. I also in general notice that if I'm just in a good mood & feeling good about myself I don't stutter even around family and old friends.