I think it feels worse when they grimace as if they are displeased at the fact that you're not fluent