At what point do I stop being nice to someone?

At what point do I stop being nice to someone? So I have a stutter and my job requires a lot of talking. I’m a cashier and have to talk on the radio and also just chat with other coworkers as well. Literally no one has ever treated me differently because of it except for one person. After I told him I had a stutter and I wasn’t anxious he then referred to me as “broken record” to another coworker and the came up and bragged about it to me. I just laughed it off cause honestly I was shocked he said that I decided to forgive him and move on since he keeps talking with me I decided to give him one more chance. I’m Christian so I believe people can change and that if they seem to have changed to give them another chance. Since it’s been a couple months since he said that I decided to uplift my grudge and forget about it. I was just chilling today at work and me and him were talking. It was seeming to go smooth I often do things to help me talk better cause it’s uncomfortable when my vocal cords close up. So I do things to help like open my throat for more air when I word gets stuck, pause, extend words/force words out, and say “um”. So me and him were talking and I said “ok I’m gonna go take this stuff back...uhh”. It wasn’t even cause I was stuttering it was just like me thinking of what to say, but I do say Um a lot for this reason. And he said “Um hahaha, I need a clicker for every time you say um. You say it so much”. And I just ignored it, I didn’t even acknowledge it. It wasn’t the fact he made fun of it that time but the fact I use it to limit stuttering. I don’t care what people think but it seems like this dude has an issue with how I speak no matter how I do unless I speak “normal”. I can’t stutter, pause, say “uhm”, etc. at what point to I stop trying to be on good terms with him and just start going off on him? He seems chill if he wasn’t...a jerk and a moron. What do I do?