I would feel annoyed. However, if someone is trying to be nice to you but they don’t know how, give them a break. If you have the patience, you could tell them more about yourself and clarify your situation. Or if you don’t have the patience, just nod and smile politely and move on with your day. Either way I would feel annoyed but choose not to dwell on it