Why is speaking on a phone so difficult compared to in person?

Why is speaking on a phone so difficult compared to in person? I am already a pretty bad stutterer in general but somehow it is MUCH worse over the phone than in person. I am trying to understand the reasons why to try and improve. Maybe it’s because they can’t see me and they don’t know what i’m doing? Or i can’t make any gestures etc to explain what i am trying to get out? If anybody else is the same i would love to hear your take on this. Also some people are soooo rude if they are strangers ‘It sounds like you’re breaking up?’ nope its just me.