I'm Sweden raised and I have the \_EXACT\_ same issue. I formulate and know EXACTLY what I want to say, and I do PRONOUNCE everything flawlessly, BUT I always get trapped in stuttering and speaking with far less confidence ALMOST EXCLUSIVELY when I speak to native Brittons and Americans. By contrast, I am very confident when I approach a non-English native and speak almost flawlesly. It's a perfectionism thing, I think. I want to sound just like they do and always assume I will fail.