Yikes, I would say if he doesn't come around within a week or so I would highly consider leaving. It feels like he's been waiting to say this you for a while and finally had the opportunity to when it presented itself. I also guess he was probably saying other things to bring/tear you down, but something like a disability is inherently off limits when it comes to most people. I've been close friends with one of my friends for years and while we argue a lot, he never has brought me down because of my stutter, so for a partner to do that, just gives me second hand anger.