The block dies down because you're most likely in a calm or relaxed state at that time. You're probably focused on what you're trying to say and communicating that to the other person. It happens to everyone who has a stutter which is why I'm adamant when I say if you can speak clearly in one context, you can speak clearly in all contexts. It's just a matter of learning how to do so