Then when the patient asks ChatGPT if the tests could give false negatives, it could provide some very valid-sounding answers that say repeat testing might be necessary.
This isn’t a hypothetical. It’s currently happening to an old friend of mine. They won’t let it go because ChatGPT continues to give them the same answers when they ask it the same (leading) questions. At this point he can get ChatGPT to give him any medical answer he wants to hear by rephrasing his questions until ChatGPT tells him what he wants to hear. He’s learned to ask questions like “If someone has symptoms _____ and ____ could they have <rare disease> and if so how would it be treated?” A real doctor would see what’s happening and address the leading questions. ChatGPT just takes it at face value.
The difference between ChatGPT and real doctors is that he can iterate on his answer-shopping a hundred times in one sitting, whereas a doctor is going to see what’s happening and stop the patient.
ChatGPT is an automated confirmation bias machine for hypochondriacs.