Stuttering comes with facial cues that many regular people are completely unaware of. Also, most regular people stutter themselves, but minimally and without the fear/avoidance behaviors. In the fact of an unexpected situation, esp when nervous,people say dumb things. I certainly have. Yes, a listener can be nervous in response to a question not being answered (did I make a terrible social mistake, do I have a giant booger dripping out of my face, etc).