I know that situation pretty well since I am a guy that was (at least until a few years ago) kind of a tall, athletic, longhaired beau. Maybe because of my stutter, maybe because of my character (sometimes hard to say) I often am not that witty or outgoing as some other guys. I can assure you there are girls around that think your stutter is cute. Did you tell her in the beginning that you stutter to take some pressure away from you? I find it helpful to do that if the communication situation allows it.