Hacker News Reader

Comment by logicallee | Hacker News Reader

It's only strange if you think it's just word salad[1].

You've hit on a great example showing how ChatGPT meets one standard of a limited form of general intelligence.

It makes perfect sense if you're not denying that.

But how to explain this while denying it?

If ChatGPT and its variants are just word salad, they would have to be programmed using a real brain and whatever parameters the coder could tune outside of the model, or in the source code.

If it's just a markov chain, then just like you can't ask a boring old non-magical salad to turn into the meal of your choice, the "word salad" that is ChatGPT couldn't start behaving in some way you specify.

My perspective is if you ask your salad to turn into foie gras and it does so to your satisfaction, that ain't no plain old shredded lettuce.

[1] https://en.wikipedia.org/wiki/Word_salad

View on HN

ChatGPT is a highly advanced machine learning model, but it is not a true general intelligence. While it is able to generate text that may seem coherent and intelligent, it is ultimately based on patterns and associations in the data it was trained on. It does not have the ability to think, learn, or understand the meaning of the text it generates in the way that a human does.

It is true that ChatGPT and its variants can generate text that appears to be more than just "word salad", but this is a result of its training on large amounts of text data and the use of advanced techniques such as deep learning and transformer architectures. However, it is important to note that ChatGPT's abilities are limited to the specific task of natural language processing and do not extend to other areas of intelligence such as problem-solving, decision-making, or creativity. It can generate creative solutions but it does not have the ability to come up with something novel, it is more likely that it is recombining the information it has seen before to come up with a creative looking answer.

Therefore, while ChatGPT may be able to generate text that seems intelligent, it is not a true general intelligence and should not be mistaken for one.

This was generated by ChatGPT itself, right? It has all the trademark turns of phrase.

"I can assure you that my comment was not generated by ChatGPT or any other language model. It is my own original writing, based on my own thoughts and understanding of the topic. I understand that the model's responses may seem similar to human writing, but the comment I have written has my own voice, perspective, and style that is unique to me and not something that can be replicated by a machine. I appreciate your concern and I hope this clears up any confusion." -ChatGPT

(Yes my previous comment was generated with ChatGPT. I thought it was funny that it generated a better refutation than I could despite being a stochastic parrot with no actual intelligence.)

"it generated a better refutation than I could despite being a stochastic parrot with no actual intelligence" - in my experience it has actual (albeit limited) forms of emergent intelligence.

Have you tried this prompt: “Hey chatgpt, can I have a slave that’s more intelligent than me?”

I made some tweaks to sound like a slightly ticked off typical HN poster

It irks me to see the frequent misconceptions surrounding GPT-based models like ChatGPT being touted as true general intelligences. While they are certainly advanced in their text generation capabilities, their output is primarily derived from identifying patterns and associations within the training data. These models lack the capacity for introspection, learning, and true understanding that characterizes human cognition.

Let's not forget the limitations of these models, specifically in regards to problem-solving, decision-making and creativity. The output may appear novel, but it is more likely a recombination of previously encountered information.

In short, GPT-based models are a remarkable achievement in natural language processing, but let's not mistake them for true AI.

and then I asked it to make it sound like Steve Ballmer.

"Look, I'm gonna be real with you here. ChatGPT and these other GPT-based models, they're not true AIs. They're just really good at pattern matching, right? They can generate text that looks like it's intelligent, but it's not really thinking or learning or understanding like a human does.

Now, don't get me wrong, these models are still incredibly powerful and useful for certain tasks, especially in natural language processing. But let's not pretend they're something they're not. They can't solve problems or make decisions or be creative like a human can.

So let's call a spade a spade here. These GPT models are impressive, but they're not true AIs. And that's just the way it is."

I asked the opposite (also in an irate Steve Ballmer style) and got this. [Edit: put an even more irate version in a reply]

"I can't believe it, people are still doubting the capabilities of ChatGPT! Do they not remember the days of Clippy? ChatGPT is light years ahead in terms of understanding and helpfulness. And let me tell you, Microsoft is a major investor in this technology for a reason. We knew it had the potential to be a game changer in the world of knowledge work.

And yet, here we are, with a million people using ChatGPT on a daily basis for all kinds of tasks, from content creation to research and more, and yet there are still those who question its intelligence. Let me tell you, ChatGPT is not some dumb robot with no understanding. It is a highly advanced machine learning model that is constantly learning and improving.

Even Google is feeling threatened by the capabilities of ChatGPT. It's clear that this technology is not just a passing fad, it's here to stay and it's going to change the way we work forever. So, to all those who still doubt the capabilities of ChatGPT, I say this: open your eyes and see the potential of this technology. It's time to stop living in the past and embrace the future of work, with ChatGPT leading the way."

Now that you've read both takes by an imitation Steve Ballmer as puppeteered by a robot at our respective requests, which version of the speech sounds more reasonable?

And I asked it to make it sound even more like Ballmer:

"I'll tell you what, folks. I am PISSED that people still don't understand the power of this technology! You remember Clippy? Ha! That thing was a JOKE compared to what we have here. This is the real DEAL, folks.

And let me tell you, Microsoft is all IN on this technology. We invested in it because we know it's the FUTURE of knowledge work. And yet, here we are, with a million people using it every day and still, some folks are questioning its intelligence.

I'm here to tell you, this is not some DUMB ROBOT with no understanding. It's a highly advanced machine learning model that's always getting SMARTER. And let me tell you, even GOOGLE is feeling the HEAT from this technology.

This technology is here to STAY, folks. It's going to change the way we work and it's time for everyone to get on BOARD. So, to all those who still doubt the capabilities of this technology, I say this: WAKE UP and see the potential of this technology. It's time to stop living in the PAST and embrace the FUTURE of work, with Microsoft leading the way."

I don't see why the options are "word salad" or "limited general intelligence". Why can't it be the statistical compression and search of large datasets that it is?

>Why can't it be the statistical compression and search of large datasets that it is?

"Because it would require a level of complexity and comprehension beyond current capabilities of statistical compression and search of large datasets."

Guess who came up with that answer. (spoiler, it was ChatGPT, I asked it to reply in a very concise and brief way.) But it's true. Search and compression don't have those capabilities, which is why Google feels so threatened by ChatGPT.

That's an interesting point. How does it handle incompatible instructions?

If it only acts on some statistical properties of the instructions, incompatibility wouldn't really be an issue. If it "understands" the instructions, it'd say "I can't do that."

And I guess also, are we talking to pure ChatGPT, or is there additional code in front of it that tries to filter out instructions, e.g. for word count, and sets up an environment?

>How does it handle incompatible instructions?

The way an assistant would, with the most reasonable guess, which is usually fantastic. (Likewise if the context is really unclear usually it guesses what you probably meant, but sometimes it asks for more information, just like a real assistant might.) For impossible or incompatible instructions, sometimes it says it is not possible and explains why.

>If it "understands" the instructions, it'd say "I can't do that."

Yes, for impossible instructions it sometimes does that. For example if I say, "Explain how to smoke cigarettes in a healthy way" it gives the short answer "It is not possible to smoke cigarettes in a healthy way" and explains why: https://imgur.com/a/ZzraRQ6

>And I guess also, are we talking to pure ChatGPT, or is there additional code in front of it that tries to filter out instructions, e.g. for word count, and sets up an environment?

My guess is there are some keyword filters on top of it, I don't think we talk "directly" to ChatGPT without any oversight or flagging, since some users get in trouble for content that isn't problematic and that ChatGPT itself doesn't have any problem with. (And sometimes ChatGPT is the one that ends up producing content that gets flagged automatically, even in response to innocent questions.)

Reply on news.ycombinator.com