Hacker News Reader

Comment by scheme271 | Hacker News Reader

The problem is how do you know whether the answer is just the most persuasive or actually the most accurate one? It's hard to figure this out without domain knowledge.

View on HN

Worse is that LLMs are trained to be persuasive by default. The "you're absolutely right..." stereotype is because these things are A/B tested on response quality and we know from studies people reliably rate vibes better then anything else - e.g. while the quality of hospital accomodations likely has some impact on patient outcomes, the view and decor of the room certainly did not fundamentally change the quality of the care provided but it is the largest determinant in how well people rate that care.

Take the output to a Radiologist and verify the veracity of the statements.

Why should a radiologist have to debunk AI slop? They have enough to do already. That's the same mentality that is frustrating open-source repositories with sloppy pull requests, and saying "here, sort this out for me".

Depending on the disease, even in cancer there's myeloma which may cause bone metastasis in many parts of the body with very focal lesions. Radiologists can't assess each and an every one of them, or even to find them all. So AI can definitely help in these scenarios.

And that AI will not be fancy autocomplete: it will be some kind of image classifier that is not trained on Reddit.

At that point, cut out the LLM and just see the radiologist.

there is often discordance between radiologists(& doctors in general) when reading the same scan(same case vignette) as well !

Do people here not realise that "second opinions" are a thing because humans disagree with each other when presented with the same case all the time? It's not just an LLM thing!

I dunno, I could see it working.

I do something similar with reviewing code: I have one agent write the code and another reviews it, then they go back and forth for a bit improving the code. Seems to yield better results than one agent alone.

Seems like a similar principle.

The difference is that in the code situation, you can run unit tests on the code, compile it, etc. Unless your LLMs are ordering diagnostics and reviewing the results, there is no further information that the LLMs have on the situation. Having a second LLM review the first is counterproductive, if the 2nd LLM is better, why not use it directly? If not, then what prevents it from sending the first on some incorrect tangent?

Also, there are multiple "correct" ways to code something, so imperfect code that solves the problem is still useful. A medical diagnosis is either correct or incorrect.

En, I think you’re just trying to justify your pre-existing position that this can’t work.

https://www.nature.com/articles/s41746-026-02619-0

https://www.nature.com/articles/s44360-025-00007-8?fromPaywa...

Different prompt approaches and training doctors to use LLMs can improve accuracy of LLM-assisted diagnosis. It’s pretty reasonable to hypothesize that LLM “peer review” could improve that as well.

> En, I think you’re just trying to justify your pre-existing position that this can’t work.

I never said it can't work. I just said that finding the correct medical digagnosis is different than finding a solution to a software problem.

Reply on news.ycombinator.com