Yes, stuttering really bothers many other people. It bothers us a whole lot more. There's that look of pity and horror on their faces, especially if you're generally fluent.