I used to have this problem as I used to imagine that people had negative reactions or expressions when I'm speaking. I've learned to block out these thoughts to try and improve my fluency. As stutterers we can't help overthink how other people react, often people around us aren't bothered by it. Eventually you should be able to speak as fluently in public as in private if you use this technique.