Anxiety makes it worse. It doesn't cause it though. If it did anti-anxiety medication would be able to cure it. Most stutterers are like you; the more anxious they are the more they stutter. But if you still stutter even a little when you're calm then you can't say anxiety causes it. In fact the only medication proven to lower instances of stuttering are a couple of anti-psychotics, not anti-anxiety medication.