Mine comes and goes. Sounds like you're better off not knowing such shallow people. Try to focus your time on a handful of good friends rather than having multiple friends of questionable quality. If your stutter is lowering your confidence, use other aspects of your life to boost it. Hit the gym a few times, new haircut, new outfit, talk to a random cute girl at the store. Just make up for it in other ways.