At least in my circle, when these people say that it's not a big deal, they just mean that I won't be judged negatively if I stutter in front of them. They're giving me carte blanche to stutter whenever I have to, and I appreciate that.