Maybe he likes you? Do you have a lot of tension when you stutter? I would probably feel a bit patronized at a totally unsolicited at a gesture like that. Especially since he doesn't know how you feel about it. If it happens again I would do what you already suggested.