Hacker News Reader

Comment by htss2013 | Hacker News Reader

htss2013

Yes but let's acknowledge the less magical reality - a big part of the magic of chatgpt is its willingness, not ability, to make stuff up and deliver it with confidence.

If chatgpt gave errors or some other bad experience instead of smoothly fabricating fake answers, a lot of the magic would wear off.

For savvy users this doesn't matter. The utility is off the charts and you can mitigate the risk of being lied to. But let's not pretend this is cost free. There is a huge externality in the form of shadow scrambling reality for millions of people who don't realize it's happening.

View on HN

> If chatgpt gave errors or some other bad experience instead of smoothly fabricating fake answers

Even better would be some awareness of what original source it has a vague recollection of, so it could say “the answer is probably at [link].” Bonus points if it fetches the link itself.

Thus far, for programming uses, ChatGPT seems to act like a bizarre search engine. It has a truly amazing understanding of my query, it’s pretty good (but far from perfect) at finding the general direction of a right answer, and really bad at actually giving a fully correct answer. I get better actual output from DuckDuckGo if a manually filter for reference material.

Unfortunately ChatGPT’s hallucinations are plausible enough that identifying them takes real work. The worst is when something is syntactically correct and works just enough one might be convinced to move on to the next problem before realizing that ChatGPT pulled parts of the answer out of its excellent imagination.

Ive been experimenting with gpt4 in a research context. I ask it to always provide a citation, that veracity is a prime directive. im fairly impressed, although sometimes it gets the details wrong, but its more right than wrong. the papers always exist. 3.5 was much worse.

This is insightful. I've been thinking that the problem is that GPT doesn't seem to have a sense of what is really true vs untrue (or conjecture or fantasy).

Maybe it does, but it isn't letting the user in on the confidence level of its statements (essentially lying to the user, as you say).

The problem is: how do you communicate confidence? If it were 30% certain, I might try to relate it to something I do know. For example: MMO drop rates. 30% would seem high! And I honestly don't know whether it is or isn't for a system like this.

If you read the technical paper, base GPT's confidence directly correlated with ability to perform problems accurately. Sadly, the hammer of alignment knocked it right out.

Wait a minute, didn't I learn in college decades ago that deciding what is true is equivalent to the halting problem? Meaning, it can't be done in principle. There might be practical ways to ascertain the truth for a useful subset of statements in certain domains, but we are a very long way from knowing the truth about arbitrary statements, if that is even possible.

"Fake it until you make it" - very human behavior indeed. It's unfair to hold the AI to a bar we don't apply to ourselves.

I expect a calculator to multiply 2 numbers for me very quickly and correctly. Every time. I don't hold any human to that bar and that's perfectly ok.

Yeah, and you can have GPT do that quickly and accurately. You just have to give it access to a calculator just like you would for a human. Tell it not to compute anything and explain how to format the output that can be ingested by a calculator or interpreter.

People have crazy unrealistic expectations of what the raw model can do but undersell what it’s possible to build with a machine that grasps language better than basically every human.

I don’t need a calculator to tell me 50 * 12 is 600. But you’re saying an AI does?

For simple calculations no, for more complicated calculations yes. Trying to determine in advance if a Llm can do the math "in its head" is a task unto itself that pretty much boils down to sampling multiple responses and seeing if it produces different answers. So it's often just easier (and cheaper) to make it use a calculator for everything and not have to worry.

It tried to convince me the square root of two was negative one the other week… It was so sure of it and couldn’t be convinced otherwise. I doubt they’ll be able to convince it to use a calculator at all. It’s like arguing with a six year old child to put a coat on in the middle of winter…

Take a look at

https://github.com/williamcotton/empirical-philosophy/blob/m...

https://langchain.readthedocs.io/en/latest/

They can be taught!

A recent paper from MS implies transformer LLMs are converging onto AGI. So it’s probably a good idea to abandon these stochastic parrot mental models.

Steve Yegge described ChatGPT as a CS grad who took magic mushrooms 4 hours ago, so they've mostly worn off, but not quite.

I'm just happy he's writing again, there's only so many times you can re-read the classics.

And in this, as in many other things, he's pretty on the nose.

I call it a disembodied politician.

The amount of willingness to confidently bullshit you while virtue signaling offense when you call it out is almost laughable.

Reply on news.ycombinator.com