I'm exactly the same way, feel pretty ok around family and close friends because they all cool about it. But since I moved to another country and every day have to speak to random people it's really hard and some times I feel like it's just getting worse, but then other days it's a bit better. Anyway I feel like the more I go out of my comfort zone and force myself to talk to people eventually I worry less about what they think, and the less I worry then less I stutter. So my advice would be just go out there and kick life's ass men, work on your speech every day and eventually it will be better.