Some people do judge us by our stuttering. There are a lot of people who stutter who struggle to get employed because of their stuttering, for example. From my own experience I've also learnt that people don't care about our speech as much as we might think. As stutterers we usually are very insecure people. I stopped stuttering some time ago, and I remember that a good friend of mine once invited me to watch a football match in his house. The last time we met was months ago, when I stuttered severely. We chatted for hours, and I didn't stutter during all that time. It surprised me that he didn't say anything about my speech. I think that he didn't remember that I used to stutter. People have their own problems. Some of my "friends" never met with me alone, only if they we were in a group. I guess that speaking to a stutterer isn't very pleasant... So I don't think there is a yes/no answer. Some people do care about our stutter, others don't care at all.