I feel that. I'm always paranoid as well and convince myself that everyone knows that I stutter so it is a pleasant surprise when they tell me they didn't notice it. Hahah your story made me laugh. It would have been funny to see the change in their facial expressions when they found out about your stutter. I try to give people the benefit of the doubt as well. There was this time when someone joked "did you forget your name?" after I had a speech block. I didn't find the joke offensive because they didn't know about my speech blocks so I know they did not mean to offend me.