Keep reminding yourself that. Most people won't be bothered by your stutter. Those who are, are not worth your time.