I came to a similar conclusion after going to an intensive speech therapy program and being basically 100% fluent for about 6 months. I was still deathly afraid of talking to people because I didn't build any social skills as a child. This realization kinda lit a fire under my ass to bring about deeper and more complicated change to my life. It's really, really difficult and can be very, very scary at times, but really the physical stutter itself is the only thing that we ultimately not have control over. We can be perfectly comfortable talking to people despite our stutters if we really want to and really put the effort in to do so. I'm not entirely there yet myself, but even despite the fluency wearing off and me being pretty much back where I started fluency wise, I've made a lot of changes not related to the stutter and I am orders of magnitude happier and more social than I was even just a year ago. I've come to realize I don't even really care all that much whether or not I'm fluent anymore. Sure it would be nice, but I'd take total confidence when speaking over totally fluent speaking any day of the week.