One advise you can consider is to tell them that you're having a bad day with your speech and that you keep stuttering throughout the day. Maybe they will tell you that it's fine or they will suggest having fun by playing videogames or some sports or they will overall be fine and will understand that when you feel fine, you will definitely come to hang out and that you do consider them as a friend and that you're not trying to avoid them.