In general, GPT will not hallucinate or give bad responses on the latter since it's working from a specific corpus of information (whether from RAG or from some context directly provided by the user). This is, of course, not foolproof since some of it is dependent on a good prompt and some of it is dependent on the question (GPT is well known to be poor at math, for example). But for summarization, it is exceedingly good.
The former is where it can tend to hallucinate and generate bad responses.
Varying degrees of enthusiasm (my father was the most thrilled) but all remarked it was accurate. Of course I did these things for a laugh because I can just access the appropriate professional on-demand.
Even for myself, there is individual variation case to case what I would want to see. Sometimes it depends on things that aren't measurable. If my brain is exhausted after a long week, or if I couldn't sleep on Wednesday, I may conduct rounds a bit differently than if I am fresh. An LLM that isn't responsive in the same way can be problematic.
What sort of experience do you have though? I'd say anyone familiar with the use of AI specifically as it related to medical records would probably disagree with you.
However, medicine and law are not areas where “probably” even at a 95% level is going to cut it. You need 100%.
(Sure a person makes mistakes as well, but we understand that and have systems in place to catch and correct. Whereas people are just tossing out LLM chatbots that are basically black box).
It’s very similar to self driving cars, getting to 95% isn’t too hard now, but that last 5% to not accidentally kill a few people here and there is WAY WAY WAY harder.