I have stuttered myself ever since I was young and now I am nearly in my late teens . I would recommend you to keep your son busy in activities such as playing with him and also talking to him ask him random questions regularly and engage with him , so he doesn’t feel uncomfortable anymore . I also have a toddler brother but he doesn’t stutter when he talks I guess it’s just me in the family