I believe when we're around someone/people we have SO much added pressure to hit our words smoothly and elegantly to avoid disappointment or humiliation from others, especially when this person has no idea about our stutter if we choose not to disclose this as if it's like concealing a huge secret. On our own, we don't have anybody to necessarily impress and there's no external factors that your mind is probably like "Even if I do stutter when I'm alone, nobody will embarrass me for it" as obviously you're in your own 'safe' place with yourself. I really recommend speech therapy though in order to at least change your mindset to not feel as though the world is ending or everybody has an issue with stutterers when it's far more liberating to figure out with your speech therapist in finding new profound confidence instead.