How can I stop caring so much about what people think?

How can I stop caring so much about what people think? Speech therapy is a little bit better this time around maybe. Anyway, I'm supposed to work on non avoidance. I've been trying to force myself to try to speak a little bit more, but man is it hard! I've gotten into the habit of never talking when I don't have to (and even when I do have to, lol). I haven't gotten made fun of that much in my life and I'm thankful for that but still seeing people's faces when I stutter the look of surprise/confusion and then pity, and how weirded out they are and embarrassed for me. Or if they're impatient or distracted or if they are trying not to laugh that's also really hard. It makes me never want to open my mouth because I hate seeing the reaction to the way I speak. Do you stop caring after a while? How can I work at getting a thicker skin? I know I shouldn't care so much what people think, but I get pretty embarrassed and feel ashamed of myself.