I'm actually really happy that he found someone like you. Like many, I am pretty insecure about it and shrug off the topic when it is brought up - pretty similar to what he did. But it seems like you've known him for quite some time .. so I think he shouldn't feel offended but rather ecstatic. I mean you sound like a genuine person but as a stutterer it does sound strange when someone says they like your "stutter" - mainly because no one really wants to stutter and many people are ashamed about it because they've gone through some bullying or rough times because of it. However, in my opinion you should just say that I don't care if you stutter or not, I really like you for who you are. If I hear this from someone (even a platonic friend), I would feel very comfortable and enjoy being around this person.