I don't understand my stutter

I don't understand my stutter I can speak very well in front of large crowds of people, I am known in my school for my speaking skills. I can also speak perfectly in front of newer people, and to myself. But when it comes to familiar crowds, like in my class, or with my friends, I struggle to get some consonants out. I don't understand why this is happening, it can't be anxiety, I don't have a natural stutter, I just don't know why. It makes me afraid to speak in closed settings, and I hate having this problem. My stutter has improved in the past few years, but before I go off to college, I just want to speak freely again.