As someone who doesn't and never has cared about my stutter, the answer is that you have to stop thinking of it as an enemy. It's not a bad thing, it's just a thing. Sure, it's unpleasant and annoying, but so is an ingrown toenail, and that isn't a big deal. It's unusual, which makes it noticeable, but no one really cares. The only people who will give you shit for it are people who are already looking for an excuse to give you shit. Sure it makes communication a little more difficult, but unless you're seriously struggling to get your point across, it's not really causing any problems there either. In conclusion, you just need to reframe your thoughts about it as a minor inconvenience that you can live with rather than a serious problem that needs to be solved.