Comment by marcodiego | Hacker News Reader

Comment by marcodiego | Hacker News Reader

It really looks like ChatGPT suffered from some form of "initial over hype syndrome". I'm not underestimating it, actually I think the application of transformers to do what ChatGPT is capable of doing is something really impressive, novel and caught the world by surprise. I'm sure we'll find many equally impressively good applications for it.

But people eventually found its limitations. And did it quite fast. People learned that it is not as trustworthy as initially thought and it is also very convincing when it is wrong. It maybe very interesting to generate texts, to startup small code for functions, to query information it has "cataloged", find some trivial mistake, make suggestions and... well, not much more than that. It may save time to boot projects, but it is not capable of managing anything larger than its "memory".

I think people are now actually more impressed by things it can't do easily. It can't play hangman, chess, tic-tac-toe... It got the phase of the Moon wrong when I asked it "What was the phase of the Moon when John Lennon was killed".

So, once people get hit by one of its mistakes or limitations, it sticks more than the "impressive part". That means, people will certainly ask themselves "Should I trust a thing that can't even play tic-tac-toe?"

I had a brief period of thinking it wasn’t as impressive as I thought, but came to realize that it’s still an incredibly useful tool for research and discovery, and occasionally debugging or loosely architecting code. I’m sure for others it has genuinely powerful and useful capabilities.

If you want it to do your job for you though, you’re going to have a disappointing time.

I almost think of it as a helpful method of finding what work I should do. It doesn’t really do any of the labour for me. Like if I want to find research papers about x, it’ll find things really well and make interesting connections between papers I otherwise might not. But I won’t rely on it to condense the papers or surface important details; you really still need to do that legwork. Same with code. It can give broad, loose, useful suggestions, but you really need to write it yourself. That’s fine with me.

For sure. It is a tool, not a person. No one gets upset a screwdriver when they need a wrench.

This is my experience. I was initially sceptical, then wowed, then stopped using it entirely when I found out how utterly wrong it was about something it really should have been able to do well.

I gave it a programming problem I was having a hard time with, and it came up with the same incorrect answer I had. I explained which part of the answer was wrong, and why. It's response was to apologise, acknowledge that part as being incorrect ... then give me the exact same answer again. I explained that it had given me the same answer, and it apologised, acknowledged it's mistake, and gave the same answer.

If any tool I use behaved like that, I would not use that tool, as the dangers far outweigh any benefits.

Was this 3.5 or 4? 3.5 kinda sucks for anything tricky.

3.5

I don't feel the question should have been difficult for it. I gave it some Rust code, and asked it how I would access certain elements of a variable. The part that made me think it's useless though, was how it was gaslighting me by apologising, acknowledging, and repeating the same answer. At that point I realised that I would have to be so vigilant when using it, that it would be less effort to figure things out myself.

> I was initially sceptical, then wowed, then stopped using it entirely...

Similar as well in my case.

I think that my problem is that the core of its answers are extremely confident, and it's personally hard for me to switch into the "doubt"-mode when I read such a confident reply.

I still use it, but very very rarely, and only when I'm desperate to get at least some hints (and I'm therefore psychologically ready to doublecheck all infos contained in the answer).

I feel it's really problematic in a bureaucratic environment. AI won't have any compunction about making a false claim. But a bureaucrat has to stick their neck out to say, yeah no that's ridiculous. And also it's easy for AI make an authoritative sounding accusation and then the system to demand the victim be able to prove a negative.

You know, people keep repeating this claim. And then you ask "now give me an example of a bureaucrat being punished by a judge, either by jail time or fine or damages" (they're, after all, mostly personally liable)

And somehow, very, very, very few such examples are available.

The reverse, however, for example Police officers getting away with sometimes literally murder, or using surveillance powers on ex-girlfriends, or beating up people without justification, or ... But equally less serious offences like government workers threatening people coming in going unpunished. That you hear all the time.

I suggest talking to non law enforcement bureaucrats. The reason you see few examples is because it's rare for a bureaucrat at least in the US to do anything outside the box.

I think you're referring to the Gartner hype cycle

https://en.wikipedia.org/wiki/Gartner_hype_cycle

People were being told it would replace massive numbers of workers—upper-middle-class knowledge workers, like us—almost instantly. That implies a very high level of cognition and understanding that anyone who interacted with it without a strong preconceived bias in favor of seeing such things would quickly see does not exist.

No, it's not Eliza, but it's also not Skynet, and it was basically being hyped as Skynet. Not by everyone, and not to everyone, but easily enough that when people see that it's way, way less than the strong AI they were promised, they're going to go away disappointed and decide it's not worth their time at all.

What they sold us was HAL.

What we got was a better MegaHAL.

Does it pass the ketchup test? Maybe it's as fundamental as people don't like being lied to.

> I asked it "What was the phase of the Moon when John Lennon was killed"

What I don't understand is this: when the AI doesn't have >95% confidence it it's answer, why doesn't it just Google like the rest of us? /s

I know this is sarcasm, but is this not similar to what ChatGPT with bing integration was doing?

I had limited exposure before it got removed, but it would actually spell out the searches it would do (often more than one) to answer your question - when it worked it was actually pretty magical.

I never did look into the mechanism for when it decided to search the web, versus just provide you an answer however.

This is what i thought, until i stumbled across ReAct prompts. The tldr is you prompt the llm to give its thought, provide it with guides for actions to take,(you do this through code) then give the answer back to the llm, itll continue in this loop. for the lennon questions

> Me: What was the moon phase when john lennon was killed?

> GPT: Thought: I need to find out the moon phase on the date when John Lennon was killed. Action: google: Moon phase on December 8, 1980 PAUSE

I didnt code this I just did the search my self following is the google result

> ME: The Moon phase for December 8th, 1980 is a Waxing Crescent phase.

> GPT: Observation: The Moon phase for December 8th, 1980 was a Waxing Crescent phase. Answer: The Moon phase when John Lennon was killed on December 8th, 1980 was a Waxing Crescent phase.

these iterative thoughts and actions really sold me on it. theres papers that go more indepth

It's amazing that the article's author talked at length about the hype decreasing yet didn't realize that this is just the Gartner hype cycle in action [0]. We are currently on the downward slope towards the trough of disillusionment after the huge peak, but over time we'll equalize towards the new status quo of using AI everywhere and not even thinking about it, same as we do with smartphones now.

[0] https://en.wikipedia.org/wiki/Gartner_hype_cycle

The Gartner hype cycle is way overhyped, and we are now in the trough of disillusionment regarding it. I dobt we'll climb out.

Why do people think the garter hype cycle is a law of physics?

The cryto people have been pointing to garter hype for over a decade. Still not using Bitcoin for anything and would be extremely surprised if my disillusionment ever subsided.

> same as we do with smartphones now.

You picked an interesting example, were smartphones ever even part of the gartner hype cycle.

I think it was just a straight steady upwards line that rounded off.

Blackberries and Symbian

Reply on news.ycombinator.com