I'd recommend therapy, if you haven't already (specifically psychotherapy). It sounds like you're having trouble with a lot of emotions around the stutter rather than the stutter itself. A psychotherapist can help you untangle those knots in your mind. It's not impossible and it's not too late. Hope you find some solace soon.