People can do all sorts of things behind your back regardless of whether you're a stutter or not. Or they can not do anything at all. There's really no point in worrying about that sort of thing. :)