I have been stuttering for 16 years and I am almost 23. For my the thing you are describing is so recognizable for me. Here is my theory on this: whenever You are alone there is no one to judge you when you are stuttering, so the fear of being different or abnormal is not there when you are alone and is there when you are with friends, colleagues or relatives.