I'm the same. I stutter less with people I'm more comfortable with too. Perhaps social anxiety is a part of it. Sometimes i think it's not that I have an anxiety problem but my body just reacts to anxiety more.