Stuttering more in learned languages?

Stuttering more in learned languages? I stutter and whenever I try to speak a language that I'm learning (be it English or Japanese), my stutter gets worse. It makes me unable to pronounce the words properly even though in my head, I can hear them how they are supposed to sound.. Anyone here have any advice?