Doctors do consider physical causes for issues like anxiety, but they’ll avoid proposing it directly to the patient in order to avoid giving them the wrong idea.
There have been several studies performed on this topic, although they tend to focus on depression. The number of cases with identifiable physical causes are in the low single digit percentages.
In other words, keep an eye out for additional symptoms that may suggest a physical ailment, but start with the most likely explanation: That the disorder is best approached as a mental health issue