Probably because they do not understand what a stuttering is and only think about it as of some sort of complete brain damage that makes you completely dysfunctional. If they keep doing it despite they know it affects only your speech then something is wrong with them.