I think it's because he's not openly stuttering. With stopping in the middle of a sentence and rephrasing it, changing words to - in some cases inadequate - alternatives, he's trying to avoid specific sounds, and that's why people think he lost his mind.