I'd avoid the phone until I have some successes in easier speaking situations, like talking to a friend. *(I've just noticed now, that you have to speak on the phone because of your job, so it's not an option in your case)*. But that doesn't mean not to expand your comfort zone (but do it gradually - learn to walk before you run a marathon). Personally, I'm fairly fluent now talking to others in person but always avoided the phone. Yesterday, I took the risk and called our city library to ask some questions and it actually went pretty well.