It really makes no sense at all

It really makes no sense at all I don’t stutter at all when talking completely alone. My stutter is purely when talking to other people. The people don’t even need to be present as it happens over the phone aswell. So what causes this? Anxiety is definitely a factor but in my case it happens when i’m 100% calm talking to my friends so I don’t think that’s it. Does talking to other people use a different part of your brain compared to talking alone? If anyone has the answer to this let me know