personally i dont like jokes about stutters bc a disability (does it count?) that affects ur whole life and how u interact with others isnt funny. if he made stuttering jokes behind ur back before, he definitely wont hesitate to do it again. the fact that they made distasteful jokes about ur stutter behind ur back is a good sign for u to distance urself from them/cut them off