If you start giving in to the fear and start avoiding the places that you fear (which will become many over time), it just reinforces to your mind that there is something to
fear. The more you can push yourself through to endure those moments
of panic, the more the opposite message gets relayed to your brain, the message that there is nothing to fear. Therapy can be a huge help with this also, learning how to handle that urge to flee, etc.