My experience has been that it depends on the severity of stutter at that moment. The severe the stutter, the more people care. Also, i know this is going to be controversial but in my observation, women are usually the ones who are more bothered about stutter than the guy. I mean, think about it, stutter is seen more in men than women, so its only logical to assume there will be a disparity in preferences between genders.