I think your feelings are valid. No one wants to be around people who are jerks. I know I wouldn’t. Unfortunately you might have to distance yourself from these people. In my whole life no one ever made fun of my stutter or even about someone else’s stutter, so I think you just ended up with some unlucky people. The good thing is they’ve shown their true colors so now you can do what’s right for you . I don’t think you’re being too serious. Your feelings matter. The world needs more people like you who have greater emotional intelligence, actually, and less people like them