I think you made the wrong choice. Better to have said “ ok, let’s be friends”. It gives you the opportunity to interact with this person, become more confident in speaking with a girl and , who knows, maybe you change her mind. If you continue to let other people’s reactions to your stutter influence how you feel you’ve given them control