Doubting myself during times that my stutter is less severe

Doubting myself during times that my stutter is less severe I've dealt with a stutter my whole life, in addition to difficulty with mumbling and leaving out sounds. It used to be bad enough that I couldn't talk on the phone to my grandma because she couldn't understand me. Over the years it's definitely gotten better than that, though the progress hasn't been linear. Right after college, I got a concussion which definitely seemed to make it worse. These days I can occasionally go weeks without stuttering, and some days it's only once or twice. I can do all the things I used to avoid. I had one friend who told me that the first time I stuttered in front of him he thought I was making fun of stutterers cause he'd never seen me have trouble with speech before. On one hand, I'm extremely grateful. I've made so much progress and I feel like my stutter doesn't hold me back as much as it used to. My stutter improving has made everything easier. But there's a weird imposter syndrome that comes with it. Because it's not like my stutter went away. There are still some days and weeks when I feel like I can't even get a full sentence out. But when I'm doing better I start wondering if I was faking it all along (as if I would sabotage myself professionally and avoid socializing some days just for fun). Am I ridiculous for feeling that way? It feels like such a silly thing to be bothered by when instead I should just be grateful for how far I've come.