Wow that is rude. However, I think you can use that as a way to try and find what your colleague who stutters does, that you may not. That is, how did his speech differ from yours, aside from the stutter?