Why girls are freaking out when they see a boy stuttering ?

Why girls are freaking out when they see a boy stuttering ? At first, when they don't know about the problem, they stay comfortable around me. When they get to know about it, they freak out and stop being around me. They even feel hesitant to talk to me after that. I want to talk to them like everyone. But this, (idk if stuttering is seen as a taboo) becomes a barrier from building a relationship with them. any suggestions how to handle this ?