He has a stutter. If he didn't lose it growing up as a kid odds are he'll have it until he dies. He doesn't stutter with you because he's comfortable with you. He's at ease with you. Your family, not so much (apparently). Keep pressing the issue and you'll keep fighting. You also have to remember that he's a person with his own desires and wants. If he doesn't want to "fix" it he doesn't have to.