Hacker News Reader

Citric acid and chlorine together produce chloroform. They are also both used to clean and sanitize water tanks (but one after the other, not together). I think it is better for the model to have this knowledge.

What I want to say, you can't simply remove this information, as it does not exist in vacuum but contains parts of and can be derived from a lot of other informations.

View on HN

It's also obvious that LLMs fall over in a vast number of software engineering contexts, when the reasoning involved hasnt been well-represented in their reasoning training data. I imagine this is a near daily experience for many engineers -- great performance one day, and crazyness the next.

So if LLMs were reduced to this pathological performance on hacking, because they'd never seen it -- and only "inferred it" -- then LLMs would be useless. As they are when asked to do quite a lot of things.

This seems to be that there's just so many degrees of freedom, that there's a pretty reasonable chance on any day that you're in a situation where nobody has been before. Or as PG put it once, my job is to think thoughts nobody has ever had before.

That being said, that doesn't mean the majority of the situations you're in are completely novel, just that there's a reasonable chance of at least one occuring.

Reply on news.ycombinator.com