It's important to remember that anxiety and panic are neurochemical responses and can have strong physical and genetic components, they don't always need to have a psychological "cause."

One thing I'd suggest, if you haven't already, is look into your family history for any for signs of it. In my case, there's a very clear hereditary line of panic disorder and other similar ailments that came down through the generations, eventually landing in me as well.

We like to think of our "mind" as something that exists completely separate from the physical reality of the brain, but in the end it's an organ like any other, and any malfunctions within it can affect us in subtle ways. For example, during the run up to a panic attack, I tend to get very irritable and easily frustrated. I've come to recognize it as a sign of what's to come and take steps in advance to try to curb it.