Honestly? Because of media representation. Portrayals of characters who stutter in TV and movies are still pretty crude and perpetuate a lot of stereotypes. Lots of characters are given stutters and then framed as weak, nervous, cowardly, or untrustworthy. Hollywood has become less stereotypical over the years in lots of ways, but the portrayal of stuttering has never really progressed. Rarely if ever do you see a character stutter despite being generally well adjusted and confident.