Hey, I'm exactly like you. In fact, I don't even try to initiate conversations and only talk when spoken to (which of course happens rarely). This leads to less people talking to me and the cycle repeats. The only way I can think of to break this cycle is to practice with family members, helpful cousins, then close friends and eventually with strangers. I often try to copy how my friends interact with new people (they make it seem effortless), but for me and other stutters that seems like a huge hurdle. No pain, no gain as they say :) I'm still trying to get better at small talk myself, but its mostly being really perceptive of the other person, while I am focused more on myself than the person I'm speaking to.