Hacker News Reader

They cannot provide information that is 100.0% accurate to the source where there is NO acceptable amount of hallucination; talking legal, compliance, etc.

They’re so so bad at saying “I don’t know” or “no results”. They’re still sycophants.

View on HN

This is true of people as well. And yes, there is acceptable inaccuracy in legal, compliance, healthcare, safety, etc, it's just very low. There's even a saying for dealing with it: "get a second opinion".

yeah but a healthcare professional won't happily give me a cocktail recipe for my diabetes insulin. and if they do, i can sue the shit out of them for malpractice and they won't be able to continue giving life-ending advice to patients.

A healthcare professional could give you terrible advice, which is why we have legal incentives to reduce harm. Now that AI companies are worried about similar regulation, the best LLMs are very reluctant to give medical advice.

the difference is the scale of harm - a single person does not have the reach of OpenAI. regulations may be coming sometime in the future, but have all edge cases been accounted for? do we even know all the edge cases where an LLM could give life-ending advice to someone?

Reply on news.ycombinator.com