We all know our fears are confirmed and true, that's what they're fears in the first place. That rule doesn't apply to every fear, but a lot of phobias and causes of anxiety are 100% justified and intuitive. Being afraid of judgement of speech as a stutterer is like being afraid of fish at an aquarium. We know we're being judged and treated differently all the time, and we are correct to anticipate it. Only thing we can do is not let it hurt our feelings as much.