No, it's a fucked up thing to say. Though I don't think he means to hurt you and it probably his shitty way of cooing with his own feelings about your stutter, possibly feelings of guilt or something, it's no excuse and he should know how it makes you feel. And then he should stop it.