It is understandable because i also have this type of trauma. It developed when i was studying in co education , and girls tend to make more fun of me as compared to boys. I am not misogynist but i am telling you guys what happened to me. So that is why i am still hesitant to speak infront of women because of my personal experience. I think that you have also faced something like this. That would be the reason that you are afraid to speak in-front of women.