As a general rule, you care so much more about your speech than other people do. Yes, there are jerks, but most “off put” reactions are just the product of people not knowing how to react. Tell him up front that you stutter, open the floor for questions, and you’ll probably find that he doesn’t have a problem. He may even find it endearing.