That's the nature of stuttering. Sometimes you stutter, sometimes you don't. The more you don't want to stutter, the more likely and/or severely it will happen. There's no magic puzzle you crack and then you're fluent.