I have the same problem, my first language is swedish. Always stuttered alot more when speaking english, which were not very often. Lately I have been speaking with a coworker from another country, after only a few hours it went alot smoother. It will get better the more you speak english, the book thing is good pratice aswell.