Depending on the context, the phrase can go both ways. If it's used to mock or humiliate my stutter, I don't like it. But if it's used in a way that is unrelated to stuttering or is humorous in this way, I'm fine with it.