I stopped training karate when I was a kid because of my stutter. I let my stutter won. I terribly regret it now, feel like shit that it just took over and made me stop doing my hobby. I think that if you stop, in a couple years from now you will feel exactly the same about learning Korean. It's not learning Korean that's the problem, it's just your stutter and OCD. My whole life literally is starting to stutter on words I wasn't stuttering on before. It eventually passes, then a new word appears. That's just how it is. Don't be like me, don't let your stutter win.