Hacker News Reader

Comment by Slighted | Hacker News Reader

Just because it can pass the test doesn't mean that its behavior will be consistently correct. Trusting ChatGPT in medical applications is the same as trusting it in programming applications: It will work for simple to intermediate tasks and may even look as though its explaining itself properly, but for specialized tasks, sufficiently complicated ones, and even in simple tasks, it can't be expected to product correct results consistently.

View on HN

I already started using it for medical applications. My friend had allergic reaction to Monster Energy drink today, and while I could find a lot of documents in Google, ChatGPT explained what I need to know, what medicine she has to buy (antihistamine, it worked), when is it important to call ambulance, what to take.

I can Google all these things, but all the SEO spam pages that I get just take a lot of time and effort.

Also she doesn't have money for private doctor, and to get to talk to a doctor takes weeks.

She was also at the emergency last time when she had the reaction, and they made her wait 8 hours there :(

ChatGPT just works for medical problems (but I still double check the answers)

Isn't all that seo spam in ChatGPT's training data?

The looming AI-generated SEO nightmare feels like we are on the cusp of a cousin to the machine translation problem.

Many years ago, someone had the fantastic idea to improve translation by comparing websites available in different languages. This shift made machine translation noticeably less embarrassing. It also meant that instead of hiring a translator to translate a site fully, you could cheaply hire a shop that runs machine translation and fixes the worst errors.

The knock-on effect here is that many websites that would constitute input data for future machine translation systems are essentially already machine translated. So we are in a G-in/G-out situation.

(source: a translator friend of mine in the AAA games industry)

If you had to consult the AI for what to do for an allergic reaction, how do you check the answer?

What do you mean?

Just ask what are the reasons that a Monster drink gave an allergic reaction and what are the treatment options.

They are asking how you would validate the medical advice given is appropriate and not dangerous.

You said “ I still double check the answers”. How do you do it?

Ah, sure, I had to do all the Google searches for that anyways (look at the caffeine content, see how it changes histamine metabolism).

Also I asked my friend if she has itching to confirm it's the same thing ChatGPT is talking about (she said yes).

Asked her if she has problem with breathing (if she needs ambulance).

Basically I used Google and ChatGPT together as tools for helping me understand her situation and the seriousness of the situation better.

It was also easier to do directed queries to Google with ChatGPT's answer.

As a comparision Google gets these kind of ugly SEO pages (the word histamine is only once in it, and very hard to find):

https://reizeclub.com/can-energy-drinks-make-you-itchy/

The main observation I've had of these AI tools is they don't learn the same way people do. You see the art generation ones which are able to do hyper realistic texturing and shading, but fail to get the right number of fingers on a hand. In the say way ChatGPT is able to answer some extremely tricky questions right while failing trivial ones.

They don’t even learn, they confabulate. “Correct” responses are an accident, and there is no notion of (especially medical) correctness involved.

60% is considered passing (and that's the range the paper says it achieves). So yeah, it's not consistently correct.

>Just because it can pass the test doesn't mean that its behavior will be consistently correct.

Now do doctors.

Reply on news.ycombinator.com