Yes. And this is a very bad habit as it can make your stutter worse. It's best to just accept it and not be ashamed about it. Most people are sympathetic to stutterers so they won't view you in a bad way or anything like that, just because they hear you stutter.