Same thing happens to me. My theory on it is that when I’m with people I like talking to, I hide my stutter less so it comes out more