One possible reason is that most severe stutterers hide and don't socialize very much, all the while mild to moderate stutterers don't stutter a lot and you don't notice their stutter unless you talk to them for hours.