You're conflating anxiety and stuttering. Which is pretty common. It's natural for one to exacerbate the other, leading to a vicious cycle. Working on one will likely have a positive impact on the other.