I think it's probably mostly due to your anxiety. In my experience, the Zoloft related increased anxiety feels different, more like a jittery, shaky, wired feeling (like you drank a LOT of coffee).
You're 5 days in, which is super...I think you should call your doctor, and ask about temporarily increasing your Xanax dose. He/she needs to know that the anxiety is severe, and pretty much constant.
As for the therapist, it's inexcusable that he's not returning your calls.