That's more or less what I've realized too, after years of psycho- and speech therapy. Now that I'm getting bit by bit more accepting with my stuttering and better at living with stuttering, I feel too that the hardest is now to change decade old negative habits, traits and patterns. At least at the current stage I feel they're much harder to overcome, than dealing with the stuttering itself. But I also believe that for this you have to reach some level of success with your stuttering through speech therapy, until then stuttering might be very well your primary issue.