I'd say it's a fear of stuttering, not speaking. And if you eliminate your fear, you won't eliminate disfluencies, but you will eliminate the things that make them awful.