As a Native english speaker I have that same experience with speaking dutch. Maybe its because I dont speak it all the time and only when I need to.