I also have this issue. I think it's because I am not trying to make a good impression and I'm not focusing to not stutter. I think it also depends a lot in your mood. When I'm with my friends, I behave diferently to when I'm talking to a stranger.