You're not in a position where you can "not call people unless it's completely necessary." I get where you're coming from – but I think you might need to find a different line of work until you can de-prioritize your own feelings about your stutter.