I think you just phrased it poorly. The guy didn't ask if you stuttered, he asked if he had stuttered. If you'd just said that you found it unfunny or disrespectful since yourself have a stutter, I think you would've been downvoted less.