Is it weird being told someone likes your stutter? Accidentally offended somebody I was hanging out with.

Is it weird being told someone likes your stutter? Accidentally offended somebody I was hanging out with. I (F18) casually mentioned that I was really into this guy's (M17) stutter. He ignored it and never really touched on it after that, but a few days later when we were talking again he said that you generally shouldn't comment on someone's stutter. I was just trying to be nice and make him more comfortable. Is there a better way I should go about doing this in the future?