The fact that the other person literally acknowledged your stutter and sympathised with you means that he is completely fine with being patient and hearing you out. You can take as much time as you need to answer or even continue engaging in a conversation.