Could the ADA itself be the cause of at least some of this hesitation by some doctors to want to deal with disabled patients?

With a disabled patient, there's a larger chance that something about your medical practice (the parking lot, the hallway, the stairs and ramps, the chairs in the waiting room, the equipment in your office, etc) aren't going to satisfy somebody and you're faced with a dumb legal headache even though you might have been genuinely trying to satisfy everybody and would be willing to provide reasonable accommodation.