I totally agree with you in the case of avoiding eye contact. I also believe thats easier, emotionally saying, to not make eye contact with your listener, as you don't have this instant feedback that your being evaluated, or someone is waiting for you to complete your speech.