Yes, take the new job! You shouldn't let your speech restrict what job you have. It sounds like the job is objectively better, aside from the added stress. I would also suggest seeing a therapist to help with the anxiety, especially if your insurance is decent. When I was working (I'm currently a returning student), I had access to therapy and it was one of the best tools I had when dealing with the stress and anxiety associated with my job.