I had the same diagnosis, as you. In my case, a 2cm IDC, no lymph node involvement, hormonoe positive.
For me, I rejected, the radiation. And the Tamoxifen. I was Peri-menopausal.
If the breast cancer comes back, I'll deal with it, then. Surgically.
For your part, you made the right decision to just stop the Arimidex. It's not gaining you, anything. In fact, it sounds like it's ruining your life.