Hacker News Reader

Comment by nico | Hacker News Reader

The chat interface + context is the magic.

GPT had been out for a while, but it only really exploded in use with ChatGPT.

Essentially, a conversational agent + tools = magic.

Yes but let's acknowledge the less magical reality - a big part of the magic of chatgpt is its willingness, not ability, to make stuff up and deliver it with confidence.

If chatgpt gave errors or some other bad experience instead of smoothly fabricating fake answers, a lot of the magic would wear off.

For savvy users this doesn't matter. The utility is off the charts and you can mitigate the risk of being lied to. But let's not pretend this is cost free. There is a huge externality in the form of shadow scrambling reality for millions of people who don't realize it's happening.

> If chatgpt gave errors or some other bad experience instead of smoothly fabricating fake answers

Even better would be some awareness of what original source it has a vague recollection of, so it could say “the answer is probably at [link].” Bonus points if it fetches the link itself.

Thus far, for programming uses, ChatGPT seems to act like a bizarre search engine. It has a truly amazing understanding of my query, it’s pretty good (but far from perfect) at finding the general direction of a right answer, and really bad at actually giving a fully correct answer. I get better actual output from DuckDuckGo if a manually filter for reference material.

Unfortunately ChatGPT’s hallucinations are plausible enough that identifying them takes real work. The worst is when something is syntactically correct and works just enough one might be convinced to move on to the next problem before realizing that ChatGPT pulled parts of the answer out of its excellent imagination.

Ive been experimenting with gpt4 in a research context. I ask it to always provide a citation, that veracity is a prime directive. im fairly impressed, although sometimes it gets the details wrong, but its more right than wrong. the papers always exist. 3.5 was much worse.

This is insightful. I've been thinking that the problem is that GPT doesn't seem to have a sense of what is really true vs untrue (or conjecture or fantasy).

Maybe it does, but it isn't letting the user in on the confidence level of its statements (essentially lying to the user, as you say).

Wait a minute, didn't I learn in college decades ago that deciding what is true is equivalent to the halting problem? Meaning, it can't be done in principle. There might be practical ways to ascertain the truth for a useful subset of statements in certain domains, but we are a very long way from knowing the truth about arbitrary statements, if that is even possible.

The problem is: how do you communicate confidence? If it were 30% certain, I might try to relate it to something I do know. For example: MMO drop rates. 30% would seem high! And I honestly don't know whether it is or isn't for a system like this.

If you read the technical paper, base GPT's confidence directly correlated with ability to perform problems accurately. Sadly, the hammer of alignment knocked it right out.

"Fake it until you make it" - very human behavior indeed. It's unfair to hold the AI to a bar we don't apply to ourselves.

I expect a calculator to multiply 2 numbers for me very quickly and correctly. Every time. I don't hold any human to that bar and that's perfectly ok.

Yeah, and you can have GPT do that quickly and accurately. You just have to give it access to a calculator just like you would for a human. Tell it not to compute anything and explain how to format the output that can be ingested by a calculator or interpreter.

People have crazy unrealistic expectations of what the raw model can do but undersell what it’s possible to build with a machine that grasps language better than basically every human.

I don’t need a calculator to tell me 50 * 12 is 600. But you’re saying an AI does?

For simple calculations no, for more complicated calculations yes. Trying to determine in advance if a Llm can do the math "in its head" is a task unto itself that pretty much boils down to sampling multiple responses and seeing if it produces different answers. So it's often just easier (and cheaper) to make it use a calculator for everything and not have to worry.

It tried to convince me the square root of two was negative one the other week… It was so sure of it and couldn’t be convinced otherwise. I doubt they’ll be able to convince it to use a calculator at all. It’s like arguing with a six year old child to put a coat on in the middle of winter…

Take a look at

https://github.com/williamcotton/empirical-philosophy/blob/m...

https://langchain.readthedocs.io/en/latest/

They can be taught!

A recent paper from MS implies transformer LLMs are converging onto AGI. So it’s probably a good idea to abandon these stochastic parrot mental models.

Steve Yegge described ChatGPT as a CS grad who took magic mushrooms 4 hours ago, so they've mostly worn off, but not quite.

I call it a disembodied politician.

The amount of willingness to confidently bullshit you while virtue signaling offense when you call it out is almost laughable.

I'm just happy he's writing again, there's only so many times you can re-read the classics.

And in this, as in many other things, he's pretty on the nose.

Personally I find completions as equally useful as the chat interface and I'm waiting for OpenAI to prioritize economizing and adding plug-ins for gpt3/4 for completions like they have for chat and gpt-3.5-turbo. Being able to rely on Wolfram for accurate completions (through the API) rather than just through chat/chat api would be very cool

I've seen a paper (I forget which, might respond below later) about GPT-4 and tool use i.e. it reliably knows when it's hit the limit of what it can do on its own, and can successfully emit commands for external APIs to handle those aspects for it (calculator, character manipulation, etc).

My hunch is if you read that, you could get "raw GPT-4" to defer to WolframAlpha via a similar mechanism (with a couple dozen lines of glue code).

Wolfram had ability to infer context for a while but never felt magical because you often hit limitations where it did not understand, unlike chatGPT

the magic was that it was free, only OpenAI can afford to do it after selling out to Microsoft for near unlimited compute resources. Google was too afraid of negative PR to do it themselves at first, they should have spun their chat bot out into a startup that wouldn't hurt their core brand. Google had an internal ChatGPT equivalent for over a year

Tried Bard yesterday, it sucks in comparison with ChatGPT. Doesn’t even translate. I was expecting a lot more from the company that made Google Translate.

Reply on news.ycombinator.com