Because they don't care, it isn't happening to them, so why would they care? You said yourself, they don't even know that you stutter, so in their mocking they think you just tripped up your words.