It can if you let it? In my experience, the initial attraction is coupled enormously to self-confidence (as you note). I think if my stuttering ever got in the say, it was just because I let it terrify me and girls could sense that. I never found any real indication that they cared about the stutter, just my reaction to it. It's easy to say "just don't let it bother you" or "own it" etc. I've given that advice on here myself, but this is hard to do. Even now, my confidence level oscillates between high- and low-levels, and I'm not dating anymore (I've had a girlfriend for 1.5 years now). I'm constantly working on it. I guess what I'm saying is 1) keep your chin up 2) it'll work out and 3) if you get knocked off your horse confidence wise after a shitty date, we've all been there. It's not as bad as you think it is in that moment.