A question for fellow bilingual stutterers.

A question for fellow bilingual stutterers. I grew up in Australia with a pretty bad stutter. That can't get two words out type. I still have it, but like most people it's often not noticed by people or it only comes out in any situation that's even slightly stressful or when I'm excited. I'm sure you all know the drill. Anyway, I've lived in Japan for the last 17 years and I thought, "Wow. I can speak this language without stuttering". Then I became fluent. That fluency cost me my fluency. Is this a normal thing? Does it happen to everyone? The worst thing is because I stutter in Japanese as well, people often mistake me as not knowing the language well. It's heartbreaking at times.