Yeah it's pretty hard to pin down. I stutter less when I talk to people that I know won't judge me for stuttering, too. I think stuttering itself is a neurological issue, but it can be 10x amplified in certain situations because we tend to tense up, think too much about the exact execution of our speech (tongue movement, lip movement etc.). Speech is meant to be a natural, intuitive process for most human beings. We (stutterers) need to apply techniques to speak better and, hopefully, integrate these techniques into our intuitive flow. I believe that, these aforementioned anxiety-triggering situations make you mess up the techniques you learnt all the time.