Is could just be me, but I find it really annoying when I'm talking to someone and they say "oh yeah I stutter too, but only when I'm really nervous"

Is could just be me, but I find it really annoying when I'm talking to someone and they say "oh yeah I stutter too, but only when I'm really nervous" It's almost insulting because I know stuttering is normal under stressful situations for anyone, even people without an impediment. I know they're just trying to relate, but they really don't know what it's like living with a severe stutter your whole life.