I relate to this so much. Ever since my stutter appeared, people have labeled me as "shy", "introverted" or "awkward"; hell, I even did so myself. But with the years I've come to the conclusion that I'm not naturally socially awkward at all, I LOVE talking and being around people and making new friends... It's my stutter that has been making me feel like I'm awkward and asocial all these years, for the reasons you mentioned.