I have a very mild stutter but anxiety intensifies it several times higher, so I would say anxiety has the most impact on worsening my stutter over time.