It could be more than the tone. Maybe some secondary behaviors like unintentional frowning or grimacing. If you're blocking on a word and you frown, that might look like an intentional angry pause, and then if you keep talking with a louder voice than before or with a frown that continues, the problem is that looks/sounds exactly like how a fluent person would show their anger. It's like "I'm not... MAD!. I'm.... FINE!" I recommend combining easing out method with voluntary stutter. When you feel the tension, because you're sensing the beginning of stuttering or a speech block, catch the tension and voluntarily stutter until you are out of the tension. So instead of saying like "I'm.... FINE!", you end up saying "I'm.... FUH... FUH... fuh... fuh... fine."