If I were you, I would ignore him until he apologizes and get on with the rest of my day. Stuttering would have already made your life miserable and you don't want someone to rub salt in it let alone your brother. I hope he gets to understand what you're going through and apologize to you.