I have similar situation with you that when I have a calm and friendly conversation I stutter a bit and when I speak I don’t stutter at all but when it comes to more intense situations like reporting to my supervisor I stutter a bit more which is kinda of annoying for me and for them