The people who I’ve known longest often say that they don’t really “hear” it. As in, it was noticeable the first, second or third time they heard me speak but not so much later on. I also had the same experience as you - I was sorta mad when people said they didn’t hear it because, I mean, it’s so obvious that I stutter and I felt like people were just saying that to humor me or spare my feelings. I sort of got over that anger when a close friend of mine told me my stutter was sort of like an accent - at first it might be obvious, but the more she heard me speak the easier her brain filtered out the stutter and just heard what I had to say. It makes sense I guess.