Hacker News Reader

Comment by joegibbs | Hacker News Reader

I very much doubt it. I think it’s a lot more likely that it’s just hallucinating due to its tendency to avoid repetition and simply choose something else. After all, why would it be programmed to return someone else’s response when given certain inputs? OpenAI would have to set that up themselves, whereas if it’s a hallucination it would be straight from the model.

View on HN

The last time this happened with titles. Everyone seemed to think it was hallucinating but it actually turned out to be really leaking other peoples titles.

Ultimately its pretty difficult to distinguish between the two, so these sort of reports do need to be taken seriously. We have no idea what sort of caching mechanisms they have with ChatGPT, nor do we know how they handle memory.

Well we at least know they use redis as a cache because there’s a CVE about it (I remember because I had to upgrade Redis on our deployments because of it)

Agreed, I just attempted it via the api and Ui and it gave a mishmash response that didn’t seem to be coherent, mostly different subjects all smashed together at the max length output.

I tried it via api two ways, both via system message and no user message and one with the word rewrite at the beginning and one without. Both gave similar long mishmashed outputs that seemed like a hallucination.

this is the answer!

This has happened since the beginning and sometimes too does happen if you ask it to repeat the same thing 1000 times

all answers are hallucinations

I don’t understand why people don’t get this. There is no difference in the process that creates useful answers and the process that creates useless ones.

It’s similar to how sometimes people just spew bullshit and then once they hear it they’re like no, wait, it’s like….

There isn’t a proponer closed loop on LLMs yet so we sometimes end up with terrible answers.

Reply on news.ycombinator.com