I know what you feel, it is really frustrating. I have been really fluent at work before, but since I came back from holiday I struggle much more again. Recently I am trying to be less perfectionist and this helps me. Rather than getting upset for each time it is not going well, I now try to set small achievable goals and be proud of them if I make them. For instance in your case, out of all the speaking situations you have in the hostel, try to pick one each day where you really try to speak calmly. Might help to disclose to the other person what you are doing at the start of the conversation. Be positive and proud of yourself if this single conversation went well and work from there (rather than being fed up about all the other situations). Next week you might have 2 situations per day which went well.