Yeah it can be challenging sometimes. For this I typically fall back on the word avoidance method. You need to roll with the punches, if you feel uncomfortable saying the word sugar in the moment (maybe it is the S), avoid saying it. You could respond with "only milk" or "milk only".