I don't see people wanting him to step down because of his stutter. I do see though people stating the possibility of cognitive decline being a real issue. Yes people will mock him for his stutter, but being the face of the US and in view of the world, he's going to get mocked for something. It's not his stutter that makes people believe he's unfit for Presidency.