I know where you're coming from but I don't think it should stop you. I purposely went towards jobs that required me to communicate with people even though it was difficult at first. By doing so it really helped my stuttering by facing that fear head on and I'm glad I did. I speak much better now because of it. Most people are very understanding in my experience. Rarely do you get nasty people but it happens. If I ever had a bad day where it was worse than usual I just told the person. I don't think I've ever had an issue with someone after I told them.