If you dont feel comfortable telling his authorities just write it on a note to them. That way they can obviously see your stutter is a bigger difficulty than what they can see. Explain even on the letter you're writing in a note instead cuz his remark really upsetted you and saying it out physically would be pretty difficult jusy cuz how upset you are about it.