Hacker News Reader

Comment by nico | Hacker News Reader

That sounds a lot like the justifications Claude and ChatGPT give when confronted about something they did wrong, or when asked to provide a customer support response about software issues

View on HN

I've lost track of the number of times Claude has basically said "it was like that when i got here" in the face of a clearly bogus choice and easily disproved explanation.

They should add a feature called "auto-really" that just automatically says "really?" after the chatbot answers a question to check if it's going to 180 upon this tiniest bit of scrutinity.

You joke but this is almost literally what Chain-of-Thought does, at least in the early days. They basically just added "Wait," to the model's output and fed it back to the model iirc

This can't be a trillion dollar industry...

"Really?"

It's the ELIZA effect.

You need to hit the retry/regenerate button more, it's there for a reason.

While the "stochastic parrots" thing is a bit overblown, IME most LLMs tend to surprisingly different responses even without changing the context, especially if they're hallucinating or doing something "wrong".

I tried asking about how my state treats violations of updoc. It replied with a long ass wall of text about how serious the Updoc statutes were in my state and how judges punish harshly.

I pointed out that updoc is nonsense and asked why it didn't catch that. The answer was that it was my fault for giving it bad info.

what's updoc?

Not much, how're things going for you?

got em

There is no difference, from the model's point of view, between code it wrote and code someone else wrote. It's all just context.

Reply on news.ycombinator.com