Yes, if taking the effort to remain fluent requires a lot of effort, no one can fault you for wanting to relax among friends and family. It makes perfect sense. No one should have to be ‘on’ 24 hours a day. Don't be so hard on yourself. Your friends and family should be understanding and you need to be easier on yourself. If and when speaking easier, it will become easier across the board and require less effort in public as well as with familiar people. It's very tiring to have to utilize all those little tips and skills in every situation. Everyone needs a time to just relax.