Perhaps it's because you now expect it to happen around people and therefore get more anxious about it. Anxiety is one of the biggest factors with stuttering. When you're with family, you may be more comfortable and confident, so you don't get anxious.