You need to get one thing straight, you're talking about communication, not simply speech when you're referring to anxiety related to public speaking. They're not the same thing, speech is only a part of communication. There's a reason why there's a thing called "glossophobia" which is the name for fear of public speaking, there's also social phobia and these are not the same thing either, even though they're both related to usually feelings of embarrassment, being criticized and usually paired with low self-esteem. And you know that none of these things are exclusive to people who stutter, so why would you assume that no one else has a reason to be anxious about speaking in public? It's not that I don't understand your point, and it is hard to imagine (for me as well) how some people can be severely anxious about speaking in public, even though there doesn't seem to be anything wrong with them on the surface. But you don't know the background of those people, you don't know what they've been going through that made them like that and you don't know what's going through their head, there's a lot of suffering in life not only for people who stutter and I feel like that's worth mentioning now and then on this subreddit, and people have all the reasons to be terrified.