Hacker News Reader

Comment by bonzini | Hacker News Reader

As of a few months ago they still have trouble, with low thinking, at the "should I drive to a car wash that is 100 m away" kind of question.

View on HN

Simply appending “check your assumptions” to the question fixed it even back then: https://news.ycombinator.com/item?id=47040530

Similarly for Apple’s “red herring” paper, simply adding a generic caveat to “disregard irrelevant factors” (without specifying which ones) restored performance even in the weaker local llama models back then.

The flaw was not in the reasoning; the flaw seems to be simply that the assumptions we make are often different from the assumptions it makes. I wonder if that might be a fundamental underlying cause of misalignment.

Low thinking is an artificial constraint. It can fail spectacularly on things that aren't in the training data.

It's a nonsensical question to ask, and how an LLM answers gives 0 signal.

If you were home and a family member asked you that question, you'd probably criticise the question rather than answering. LLM are RLHF'd into being milk-toast helpers that just try to answer questions like that with no criticism.

This is all beside the fact that the world of AI has changed pretty dramatically in the last few months.

*Milquetoast

TIL. I feel like I've learned this a few times now, so we'll see if it sticks this time.

It is so nonsensical because it has such an obvious answer. The answer is so obvious, in fact, that one answer can be considered nonsense and the other common sense.

I disagree pretty strongly. If someone asked "Should I drive to the carwash?", the most obvious response, and the one nearly everyone would give, is a question: "why are you going to the car wash?" because asking the question implies you don't need the car with you.

That's not what was asked, GP was summarizing. Two options were given. With one option you wouldn't have your car with you when you arrive.

The only context that these LLMs have is whatever is in their context window. Not only that, the context is updated on each token emitted with the new token. It's a chaotic process, and a flaw inherent to the technology. It's not going to be fixed anytime soon.

"The car wash is 100m away. Should I walk or drive?"

Everything I said applies to that question as well. The only difference is that'd I'd probably also ask "why are you phrasing that question so strangely?"

This is just a stupid post.

It’s nonsense to test if a product that is marketed and sold as being able to provide generalised intelligence on demand, does what it says on the tin?

Check yourself

Since you're new here, I'd suggest you read the guidelines for etiquette.

https://news.ycombinator.com/newsguidelines.html

It's very unlikely that person is either new or unfamiliar with the guidelines. They almost certainly created a throwaway account specifically because they know the guidelines and want to flout them without consequences. (It seems like there has been an uptick in the number of these kinds of throwaway flame comments. I wonder if HN tracks that?)

Reply on news.ycombinator.com