What your grandfather said to you isn’t right at all, he came from a different time is a big part of why he said that, I’m sorry. I don’t have much advice I’m in similar shoes I don’t have a terrible stutter but it does effect every single social situation that presents itself. I think its one of many reasons that fuels my drug addiction, but stuttering definitely stunted me from a social development standpoint when I was a teenager and even now.