I have related closely to what you say. Voluntary stuttering has really helped me with this. It really helps desensitize you and decreases the fear in those moments. And never, ever apologize for your stuttering.