There are too many qualified doctors in the major metro area I live in to put up with ones where I can't quickly and efficiently make an appointment, can't get an appointment in a timeframe that is useful to me, and where their staff can't seem to successfully process routine insurance claims with major insurance providers.
Also, randomly billing me for stuff weeks or months after an appointment with cryptic explanations thoroughly pisses me off -- even it if is only $15.00 (going to get a new podiatrist over that one...).
As for actual "doctor specific" reasons for leaving a doctor, not explaining the reasoning behind his/her recommendations for me to get certain medical tests or procedures done will get me to look for another doctor. I don't actually need to fully understand the reasoning (I'm not, afterall, a doctor), but I need to know there is a reason beyond "hey, you said your elbow hurt and I think I can get the insurance company to pay for an MRI!".