That's a cognitive bias. I would say most people definitely aren't assholes and don't laugh at stutterers.