Yeah! I think you're really good at interpreting this stuff. Maybe they also have a bias towards that emotion because they're unconsciously trying to empathise. So they're thinking "if I were struggling to speak, how would I be feeling?" and they're basing their reaction on their guess of what your emotions would be, but they're also looking to you for confirmation of that guess (and generally to see how you're feeling). If they see you looking relaxed and comfortable and confident, they're definitely going to relax too.