Yes, thats exactly what you need to tell her, how you feel about the conversation and ask her these questions. It may be weird but it's better than feeling bad all the time when you talk with her. For me personally, I hate when people look away when I stutter. They look away because they probably feel awkward, but for me it's kind of hurtful. I want to tell something to somebody and they just look away like they are not even listening to me?