Depends on a lot of factors, but i'd say that your safest bet is telling them the truth. Tell them that you have a bad stutter and that it is particularly severe when calling (you can maybe even make up smth like it being due to you not being able to see your interlocutor's facial expressions) and that you would be more than happy to meet them in person.