What helps me is doing a short speaking exercise on a daily basis. The exercise should be something that's easy enough for you to sound fluent but challenging enough for you to feel you're bettering your speech. And, every time you stutter and feel bad about yourself, you think about your last exercise and remind yourself that your speech can be a lot more better and that you're working on your speech! I believe you can have/gain confidence when you have stories of little success! It's like building the foundation of your confidence with tiny blocks of your little success stories. Do some speaking exercise everyday (it can be something as simple as reading a page of a book out loud and recording yourself)!