Many women don't mind, including my own partner. When we met, she saw it as normal. That said, try to work on reducing the stutter so it doesn't impede your speaking too much. I really loved the book "Beyond Stuttering"