This might not be applicable to you but I find that being anxious/nervous is what makes you an easy target so to speak, by projecting confidence and conveying that you simply do not care or are not shaken up by a comment of an irrelevant person in the first place you make it a whole lot less attractive for people to roast you. By giving in you are basically giving them the attention that they are seeking.