A language barrier makes stuttering worse.

A language barrier makes stuttering worse. I'm 15 and got my stutter around 4 years ago. I am half french and my dad lives in France whereas my mother lives with us in the UK. My dad has always wanted me to speak in French to him as it is his main language but ever since my stutter came around, I have been struggling to even completed half a sentence in French. While speaking English, I have a mild stutter which annoys me but my stutter in French is a thousand times worse. I see my Dad look dissapointed, upset and sometimes annoyed by my stutter that I just have to speak English (which he barely knows) just to speak to my Dad. I can't take talking to him like this, is there any explanation or help you guys can give me? Thanks