I would honestly just ask or tell him. He might be looking away cause he doesnt know what to do in that situation or he assumes you dont want him to look. I have physical behavior to so i gotcha ik how it feels. If he still engages with you, i think ur fine. Everyone is a little weird so all good. As far as the jerking off the head, I have that to but ive been actively working on it. At least for me it wasnt a subconscious thing, like i KNEW i was doing it. I just tried to push thru. but down i actively basically say "wtf" in my head and stop. Not saying you have to stand still completely, but ACTIVELY pay attention to what ur doing when you stutter, whether its you looking, or someone telling u shit they noticed.