I struggle with both. There have been times when I knew the answer and chose not to speak up due to my stutter. There have also been times when I could not avoid a particular situation and my stutter wasn't manageable. Those bad feelings about disfluency will be there regardless of how well I "pushed" through it or got my point across. I am working on accepting it more and coexisting peacefully with it. Perhaps you can relate.