I don't think it's your stutter that makes people nervous or uncomfortable. I have a severe stutter. When I struggle a lot and feel frantic and ashamed while talking, other people feel nervous and uncomfortable for me. It's because they can sense my feeling and how hard this is for me (the emotional side, not the actual stutter). The less reactive you can become to your stutter, the less other people will react. Most don't have a clue what stuttering is like or how it feels so they look at us to know how to react, aka if we're nervous and panicked then they are going to feel nervous and uncomfortable. You probably won't grow out of it. You can try to control it (I really don't recommend this/focusing on fluency. What else you can do is learn to manage it so that you can stutter without it being so painful. My advice is to find a speech therapist who specializes in stuttering (very important), aka board certified in fluency disorders and make sure they don't focus on fluency or techniques.