Some people just have a listening face that just looks like they are impatient. And some of us frown a lot when we are trying to get through blocks. If people we're talking to do not accept our explanation that our frowning does not represent our feelings, they will try to "save" us by giving us less speaking time. They assume we are making a "i don't want to speak" face when it's actually a Marathon face. A good way to see if they are actually being impatient is think about how often and how early they interrupt you mid-sentence and how they respond to you when you say "let me finish my sentence."