With the proliferation of AI note taking, this advantage purports to have been democratized, but I'm not quite convinced. Since AI summaries are far from infallible, a mistake is bound to sneak in here and there (note that these aren't mere transcripts, but summaries split into sections). The provider may or may not go in and clean up your AI notes afterwards, any mistakes made by AI are effectively disowned in terms of responsibility, and admin will still pressure providers using these note takers as leverage to be able to see more patients than otherwise possible (admins want to see both lower costs and higher patients seen per day).
When you refuse this type of service, you're demanding a higher bar for your notes, but it comes at the cost of a distracted provider (who has grown accustomed to AI note taking, and only has so many hands, so they have to go back to the keyboard every now and then after checking your body).
In summary I think it comes down to how much you care about note quality versus care quality, which is likely different per person. I don't have any allergies, am not on any medications, and generally only go in for routine checkups, so in my case the notes are more or less a bureaucratic requirement that I'm happy to do away with cheaply. For others this may not be the case, and having quality notes may be critical to their care, in which case they should definitely refuse AI scribes.
Of course, none of what I said goes into privacy, which is a significant matter. However, "iPad scribes" which are essentially third party contractors remotely taking care of the notes already exist, so those concerns which arise even without the use of AI are a bit of a different topic.