ChatGPT Hype Is Proof Nobody Understands AI | Hacker News Reader

ChatGPT Hype Is Proof Nobody Understands AI

nabilalouani.substack.com

These articles are all the same. It's like complaining how the Ford Model T couldn't drive you to England in the early 20th century while ignoring how it would revolutionize cities and life within America.

Whether you believe it's 'real' or not is irrelevant. It's here, it's useful and the world will not be the same as it was.

What about the fact that it's not reliable? Seems unusable to me if you are not a domain expert that just gets help from the system and who is able to judge whether its advice is true or made up.

It’s reliable for many many use cases (e.g. anything that transforms text into a different representation), but not for a few others (e.g. anything that requires logical reasoning) - a F1 car is great on race tracks, we don’t see people taking them on gravel roads and then complain they’re unreliable …

I'd say it's unreliable if you can't decide apriori for which use cases it's reliable. For instance take this example where it makes contradictory assertions

https://news.ycombinator.com/item?id=34288209

How reliable was it a few years back? Oh yeah, it didn't exist! AI is improving at breakneck speeds and will only get better

Humans aren't reliable either. This is why we write tests, have code reviews, fact-checking/editing, etc.

And if anything the LLMs are better than the average person realises. Everyone I have demoed it to assumed it was like a slightly better Siri or cleverbot. None of them expected it to be capable of synthesising full long form text works with coherent and largely accurate content.

Language models have been around for a very long time. However, they were much smaller than today and were mainly used to check the fluency of a text, basically if a text was a problable text in English or in French for instance. These models were trained over words and had a huge problem with unknown words or spelling errors. They were used in translation models to check the quality of what was produced. In this sense, a language model might seem something ancient. However, if we take your analogy with a car, modern LM are to ancient LM what a plane is to a car. You still travel with it but it is a complete different beast. To say that these models are nothing new is a total travesti of reality. Transformer architectures have been around since 2017, a mere 6 years, but they have revolutionized the way LLM are trained and implemented. If you speak about language models without talking about transformer or attention, you are basically saying that a plane is the same as a car.

How would LLM revolutionize everyday life?

Sigh. This guy can chirp about how "ChatGPT doesn't understand what it's been asked" as much as he wants but he is missing the forest for the trees.

I'll use myself as example. ChatGPT can write better than me, it can summarize better than me, it can research quicker than I can, and it is more creative than I can ever be.

I don't care if it "understands" in a way that would satisfy Nabil Alouani, its ability to produce useful output is a game-changer.

Yeah boring, god of the gaps.

Imagine being the 10000th person to give this uninspired take.

> Cats have a representation of the world that help them navigate it. They also have instincts programmed into their brains, allowing them to act with relative logic. In contrast, GPT and Co only have patterns of human-written text devoid of meaning.

This is wrong. To say chatgpt doesn't represent the world instead it only has patterns is distinction without difference. This is the new dualism. There is "real understanding" and AI can't do that by definition because "understanding" isn't anything except being a human and acting apparently.

The Loki thought experiment is laughable.

It's always tech-adjacent types that drop these crap takes from what I've seen. Maybe they're threatened as SWEs are last to go when LLMs can generate code - someone has to check it; we just won't need two PMs, scrum lord and a BI attaché for that...

ChatGPT has allowed me to explore programming topics where there is, otherwise, very little useful information - it won't produce good code verbatim but it doesn't have to - it's my robotic rubber duck.

If there is otherwise very little information then where do you think ChatGPT is pulling that information from? It does not pull stuff out of the void, it's only as accurate as the data shoved into it and that data can be very wrong.

People get into a state of false confidence using ChatGPT for things they cannot easily fact check.

Absolutely, it's comically wrong most of the time but it's still a useful tool to solidify understanding in a way that's hard to do organically.

Was this your opinion or ChatGPT’s?

The fact that you're asking that sort of proves my point. (I know you're probably trying to be funny though)

The amusing thing to me is the article just made me think of the people in the 80s that would argue rap isn't music but just a form of noise and how rappers aren't even singing!

I actually don't understand how someone thinks we understand understanding well enough to write this article.

> ChatGPT can write better than me, it can summarize better than me,

Probably. But you still need to give some input about what to write and proof-read the output.

> it can research quicker than I can

I don't think so. AFAIK, it doesn't research at all right now. When google and bing add such models to their search engines, I doubt they will be able to research quicker than a human mind. Most research queries are not responded by plain text: Search for a hotel price in a specific room for a specific date, for instance, or check the weather in a specific place at a specific time. Language models will help you to ingest and query large volumes of text, but that's something we don't do that often.

All in all, Chat-GPT is a tool to work with text. It certainly can be useful in that context, but I don't see any world-changing properties.

I've find two groups of people on this subject, first one has taken a graduate level stat course and/or ML course, and may have work experience in machine learning/data science. The latter camp is more numerous and did not do those things.

The latter group are far more hyped about ChatGPT et al, despite explanation by the first group.

Don't get me wrong, ChatGPT is very exciting, just not in the way it is frequently portrait to be, in particular, development on this model will not lead to General Intelligence, which is not a data training/stat problem altogether. With how the ML field is shaping up to be today, it doesn't even look to be a Machine Learning problem.

> first one has taken a graduate level stat course and/or ML course, and may have work experience in machine learning/data science

I think it's just a few people getting salty because they understand "how it works" but don't understand "why is getting popular". So they start to shout from their ivory tower without realising the tectonic shift under way.

Sort of, you know how to build a stupid mobile video/photo app, but fail to make it popular like facebook, instagram, tiktok, etc. The tech used is certainly not transformative on the surface, but the implications of mass adoption and impact at a societal level are huge.

I for one I'm excited about its usefulness and how this is going to speed up the AI race. The genie is out of the bottle.

I'm also very excited about how useful it is, after years of mostly academic papers and poor industrial attempt to utilize it, there's finally something that truly showed the power of the LLM model to the masses. It's the iPhone 4 of language models.

But iPhone 4 isn't fusion energy. It's incredibly useful and a tectonic shift in the cellphone industry, but it's not producing energy. The analogy might sound completely insane, but that's how different machine learning of today and general intelligence is.

This will replace humans in certain jobs - just like McDonald's touch order machines and 100000 other times when a machine of either digital or mechanical contraption replaces humans in history. This is not however the beginning of singularity where the AI gets a mind of its own and start a symbiotic or adversarial relationship with human civilization, this isn't even a step in that direction, this is closer to the internet or wikipedia than that.

Well explained. The analogy is on point

Excellent point about "why its getting popular".

It's getting popular because a bunch Vulture VC's see a huge oppotunitiy to pump and dump.

They're gonna make billions! Who gives a shit if it'll really solve any problems, or make real revenue. It's immediate primary value is in it's ability to create stock value inflation.

Because shareholder value is really the meaning of life, right?

I think there's a further split between "academic experience" and "real world experience".

I'm mostly a developer to be honest, but my job title is technically "data scientist". From my experience most stuff I see is not a math problem, it's a data problem.

You want to optimize $thing? Well, you'll need a pipeline to fetch the necessary data and have them ready on demand, and that's actually harder than modeling.

This isn't a dunk on math or research; I'm a math fan (?) if anything, I just find a large part of the discussion around ML to be very far removed from the reality on the ground.

Indeed, I consider ChatGPT to be extremely impressive on an engineering perspective. The data science team I've observed have a difficult time to bridge science and engineering, it's hard to find someone proficient in both, despite all being very smart people.

“development on this model will not lead to General Intelligence, which is not a data training/stat problem altogether”

Why is everyone so damn sure about this? Not to pick on you—I’ve read so many self-backpatting comments that feel the need to reassuringly say something like: “we all know this isn’t AGI and couldn’t possibly be and won’t lead to it.”

I don’t think this is so clear cut.

It is because of alan turing. The turing test has arbitrarily put the bar at “must successfully emulate a human being to be labeled general intelligence”.

I don’t find the turing test useful at all as a benchmark. We don’t need something to successfully emulate a human to do useful intelligent work. ChatGPT can already do that work in many domains and many languages, if working as an assistant to a human. I honestly don’t know why we shouldn’t call that general intelligence.

The argument “but it doesn’t understand” is not useful either, and it is the one I read most from people who deeply understand how it works. Many (most?) people don’t really understand what they do for a living either, they’re just going through the motions and need supervision to prevent then going off the rails. Just like ChatGPT. And still someone finds their “intelligence” useful enough to pay them for it.

I can see why not believing Turing Test is a useful benchmark means that you can't deduce ChatGPT is sentient from it being able to pass one. But how do you deduce it is not AGI without some other reasonable benchmark? Moreover, how can you deduce GPT-4 can't spontaneously become one even in principle without having that benchmark? Those two are the questions to the skeptics really.

Right. I don’t see why human-level AI is the criteria for AGI. I think we can have weak AGI, potentially long before we have strong AGI. We need clear benchmarks, otherwise it turns into theology.

Because at the end of the day its still a bunch of linear algebra optimized to predict the next “word” that would fit into a sentence. This is very evident because you can convince it of anything because it desperately needs the conversation to make sense.

And why would mathematics not be a road to intelligence?

Because general intelligence don't arise from statistical analysis of extremely large amount of data.

Think human child, or even adults. They don't know near the amount of data that ChatGPT has been trained on, yet react far better when they are put in a situation that they have never seen before.

> Just like clients have an edge over waiters, OpenAI’s CEO had an edge over his listeners — and what he did was take every single one of them for a fool.

It's funny that the author writes about Sam taking people for fools and then treats his readers as fools

> Large Language Models don’t understand anything

I'm not sure who the intended audience for this is, but I'm fairly certain HN readers don't think LLMs "understand" anything

I do! So long as we don’t appeal to consciousness as the basis for understanding but measure it with benchmarks, LLMs understand many things.

Maetschke, S., Iraola, D. M., Barnard, P., Shafiei Bavani, E., Zhong, P., Xu, Y., & Yepes, A. J. (2021). Understanding in artificial intelligence. arXiv preprint arXiv:2101.06573.

I saw this example on Twitter today which shows ChatGPT doesn't understand:

Mike's mother had 4 kids; 3 of them are named Luis, Drake and Matilda. What is the name of the 4th kid?

I'm sorry, but I don't have information about the fourth kid. The information you provided is that Mike's mother has 4 kids, and three of them are named Luis, Drake, and Matilda. However, the name of the fourth child is not specified in the information you gave.

But thanks for the paper. I'm reading as much as I can.

I perceive that as: misunderstood a riddle, but understood the language

I had to read that 3-4 times before I understood that it was a riddle. So I put it in GPT3–and it got the answer right away:

“Mike's mother had 4 kids; 3 of them are named Luis, Drake and Matilda. What is the name of the 4th kid?

The fourth kid is Mike.”

So, maybe they just needed to try again.

It's not a riddle because the answer is plain to anyone who understands English. I'm not saying you don't but the answer is right there.

I asked it a followup:

The name of the 4th child is in the question

Ah, I see! My apologies for missing that. In that case, the name of the fourth child is Mike.

I had to read it multiple times before I understood why it was a test of understanding. It is a riddle.

even if you classify it as riddle, you still need intelligence to "solve it", which is what is missing in LLM. the good answer you later got was probably prompt created by someone manually, to further feed the model.

did you need more data to solve the riddle ? rather more attention to what you read than more data, and that's the sole issue with AI/ML in current form.

Q: A week back I had a weak back. What should I do?

A: Something went wrong, please try reloading the conversation

I couldn't get ChatGPT to answer despite multiple tries. It answers simpler questions just fine

I've tried a bunch of riddles, and it often cops out and refuses to try, but you can nudge it by telling it is a riddle and trying to nudge it. Sometimes you need to lead it a bit the way you might with a human who can't find a strategy to work their way through.

This doesn't seem too different from dealing with people to me.

It might not be all that smart, but if you try leading it through riddles step by step it's if anything harder to dismiss its ability to reason and understand - spitting out an answer wholesale might just demonstrate recall. Being able to talk through the steps, and getting it when you suggest it answer a sub-problem first, is of anything more impressive to me.

Where’s the article about this example ?

It doesn't show it doesn't understand. It shows that if it is capable of understanding, it's reasoning skills in some areas lag its language, and that is confusing to us. Plenty of people would fail to answer that riddle without prompting, but they'd likely answer in simpler language, and so we would evaluate them accordingly.

I asked it a bunch of riddles yesterday, and it failed to get many at first, until I led it through parts of the reasoning the way you'd lead a child.

E.g "you leave your camp, go 3 miles south, 3 miles east, 3 miles north, and find a bear in your tent. What colour is the bear?" At first it said it couldn't tell. I then asked it where the tent must be for the description to be true, and it correctly answered the North pole. I then asked what colour the bear must then be, and it correctly got the answer.

If I had to lead a child through it like that, my reaction would not be that the child can't understand. But when ChatGPT responds in formal, adult language it makes it hard to give it the same benefit of the doubt.

I don't know how much we can say that it actually understand, but I also think it's too easy to dismiss it like that. It is also too easy to assume based on other examples that it is "smarter" than it is.

Accurately evaluating understanding is hard especially because it's ability to solve problems is "lopsided" - it's unusual to be this dumb while at the same time being so eloquent and "well read".

I asked ChatGPT this same question and it said:

> Mike is the fourth child.

Is it training itself continuously?

> Both Google and Microsoft are pulling the same shady technique [...] They will ship free bullshit-generators and say it’s up to everyday people to sift through the mess.

Can't really understand the meltdown some people are having... Honestly, what you get from well trained GPTs doesn't seem that different than getting search results and and sifting through human-generated bs yourself... including biased mainstream newspapers, stupid StackOverflow answers, self-made wikipedia articles, opinionated blogposts, and a long etc that you need to double-check anyways.

That probably makes LLMs more human xD, at least at this early stage. They should get better with time (hopefully).

Someone page DesCartes. How do I know everyone in the world is actually real, actual intelligences and consciousnesses distinct from my own? All these articles lack a severe lack of imagination or just try to apply a very narrow definition of intelligence. It's good to be grounded and not hop on every bandwagon but to deride ChatGpt because it can't convince the author it is mimicking the way we think we think is a bit daft. ChatGpt for me has been amazing - I use it to get started. My biggest stumbling block, at anything, is getting started. ChatGpt can get me started by generating boilerplate code, etc... and yea sometimes it is not quite right (though more often than not, it's pretty damn close) and guess what, it did it in seconds. It would have taken me hours to get to that same 'not quite right' level. ChatGpt is an elevator for that initial step function of resistance to my productivity. It gets me going when motivation or the willingness to focus is not there. They can charge $20/month, they can charge $100/month, I'll happily pay it.

(edit: they just launched the plus for $20/month - shut up and take my money)

your experience absolutely matches mine with the getting started hurdle. and yeah i just subscribed today

One might argue that intuition and understanding are simply complex cognitive processes that occur within the brain. In a similar vein, I think ChatGPT will evolve to understand and intuit through integrating with additional advanced machine learning models. Think of ChatGPT having side-car machine-learning models guiding it and having it edit the outputs as it writes. Similar to how we correct ourselves or rephrase something. I don't know what these look like but maybe they would be models for cultural norms (select your 'culture'), social norms, math, sciences and morality. Weighted decisions are made based on "values."

Cognitive processes as we think and speak are constantly refining our thoughts and understanding. Refining our outputs. I believe that is what comes next to advance ML. There might be a lot of hype right now but ChatGPT is a reminder in how far we have come and a small preview of what the future looks like.

Disclaimer:

  - I don't know anything about anything. 
  - I already find ChatGPT useful. 
  - I might be biased, I was a Loopt user as far back as 2007. I will buy about anything sama sells.

I agree that chatGPT et al are way overhyped, and don't represent any fundamental advances beyond product development.

That said, this article's a bit of a strawman, hinging way too much on a pedantic definition of "understand". He's correct, but it's not really the crux, and unfortunately this will all be very easy to dismiss

I don't really agree that much with this idea, "..any fundamental advances beyond...", ok, technically chatGPT is just somewhat a reshuffle of pretty well known models, but the implementation is king, always.

And first, some weeks ago you have all these powerful secret models behind corporate firewalls, inside some secure lab, now most of those owners are rushing to reshuffle their respective models to catch on with chatGPT and now with Bing + chatGPT.

Hence, the implementation is what did the trick, the tech was there for sometime already (more than a year? I'll ask chatGPT). And now other public facing applications of this technology are on the table too.

Because, the rabbit is out the hat already, if you don't use your technology, somebody else will have the upper hand in other reachable markets, just like Microsoft is now running away a good million miles ahead of the rest of the Search market competitors.

The author says: “Large Language Models don’t understand anything — and they can’t get better at something they can’t do.”

Yet, LLM’s can understand (unless one claims that understanding requires consciousness): https://arxiv.org/abs/2101.06573

And, strangely enough, LLM’s can get better at something they can’t do.

https://news.mit.edu/2023/large-language-models-in-context-l...

https://arxiv.org/pdf/2211.15661.pdf

> Models don’t understand anything

Are people really so unaware of themselves? What is understanding? I'm not going to even attempt to define that, but I will posit that the only way we have of verifying understanding, regardless of what definition you use, is validating the correct output for a given input. For instance, when we teach children, we validate their understanding by giving them a test. Are we able to look into their brain and pinpoint exactly where that understanding comes from? No. So why do we apply a different standard for an algorithm? Clearly LLMs understand a lot. That doesn't mean the algorithm is conscious or sentient, but it has understanding.

I'm pretty sure that ChatGPT doesn't "understand" things as humans do.

But does it mean that it has no ability to understand things? For example, assume there is an alien species that can only see ultraviolet, but not any other lightwave in human's visual spectrum. They don't "see" things as humans do. But we'll still consider they to have ability to see things.

Philosophically, it matters whether LLMs understand something or merely act like they do. Practically, it doesn’t matter.

Per chatGPT

"According to Wittgenstein, understanding is not just a matter of having the right information or knowledge, but also involves the ability to use language in a way that is appropriate to the context and situation."

that would mean a gear has understanding of torque?

I don't know how I feel about this article, especially around the definition of understanding.

I have two children under 6, and if you watch their learning patterns they follow a very much the same path, and mimicry is part of that. They learn the subject matter at hand through repetition, and that then build comprehension which is the basis for understanding.

I think ChatGPT has learnt and started to understand language based on the data given to it, it has built some understanding how language is structured as it is able to regurgitate language.

It has contextual understanding at some level and contextual awareness. One of the next logical questions would be, does it have intelligence and that I don't think is the case. It understanding is based on its understanding of the language.

I think the bike example is a great example, my son is learning to ride a bike. He has no understanding of balance as the article states, he has awareness of it, he knows he needs to peddle to move forward but he has no understanding of inertia, he has to steer but he has no understanding of physics, but he understands all of these components are required and he knows that via data "the experience of learning". I think it would be difficult without prompting to get him to explain his "understanding".

Now understanding is great, you can apply learned information and apply it to another subject. So we can use what we have learned from ridding a bike and apply it to a motorbike. I am unsure if this cross domain transfer of understanding is something which ChatGPT can do, but it does a great job of extrapolating data and apply it to learnt rules.

People who can’t understand ChatGPT hype would have not used calculators when they came out on the grounds they didn’t understand mathematical proofs.

like the high school teacher who insisted multiplication be done with a slide rule?

I wouldn't have used a calculator that sometimes miscalculates things and claims them to be correct.

Calculator that computes that 3 + 2 is 8 should indeed not be trusted much.

Do you know any such calculators?

The article doesn't understand large language models. It makes an assumption about how they work and uses that assumption to draw a conclusion.

Most likely, as found in small models [0], large autoregressive transformers understand machine learning algorithms well enough to apply them to their inputs to achieve in-context learning (few-shot prompting) by performing gradient descent within the overall network [1]. This is fundamentally different from static predictors; LLMs are dynamic modelers that learn from context in order to better predict tokens.

[0] https://arxiv.org/pdf/2211.15661 [1] https://arxiv.org/abs/2212.07677

I find it amusing that some people expect factual responses from ChatGPT.

Meanwhile, my script writing friends are having a heyday, and Hollywood is set to green light several new all GPT-enabled scripts.

Ad copywriting is forever changed.

And GPT codex can generate Javascript/Java boilerplate faster than you can search on StackExchange.

Gary Marcus has been incredibly vocal against LLM for quite a while now. The problem is that the guy does not understand that modern LLM are no longer just classifiers, as the article seems to imply. The underlying transformer architecture has completly transformed the field. For instance, training an AI to recognize cats has nothing to do with training a LLM, texts are fed to the machine without any labels. Attention is where these models take their strenght. It is a way to learn how to predict the next token by learning which elements are connected between the input and the output so far. These architectures model a sequence, which is very different than simply finding the next token with a basic probabilities as ancient approaches did.

I think the real innovation that underpins chatgtp is not the LLM but the UI. The large companies have finally woken up to the fact that a simple chat interface is superior to most people than having to filter and scroll a bunch of Google links and description text - it removes a barrier to the original intent (find out the answer to X). It’s value is utility and as long as it’s correct 80%? Of the time - people won’t care. It’s the same phenomenon as when you are working on a tech team and are new. It’s easier to ask the guy who’s been there a couple of years where a piece of code is rather than go hunt for it yourself, even if he doesn’t always know the correct answer.

>If you want to know what a person is really like, watch how they treat the staff in a restaurant. Power reveals the true character of those who hold it.

This is the only part of the article I am in total agreement with.

The rest is just another variant of the Chinese room argument (https://en.m.wikipedia.org/wiki/Chinese_room).

My own take on this is that people are searching for the magical in the wrong place ("we are so special") rather than seeing things are magical by their very existence

ML and AI has been one of the least overhyped things in the tech industry. Chatgpt is the fruit of a lot of that underhyped work and the acclaim it's getting is based on real merit unlike crypto

it will be the next financial bubble like dot-com, im calling it

Hey it's the author, Nabil Alouani, and I have something to say to all the unproductive (and often toxic) feedback: KEKW.

Shoutout to the one who caught the "Indian" characters mistake. I just corrected it. Thanks!

have people not seen the videos of Gato? it's pretty mind blowing. it's ability is beyond belief, and it's not even the scaled up version, since there have been clear demonstrations that scaling improves model performance.

to me, Gato is pure proof that tranformers are waaaaaay more capable of emergent understanding than people seem to be willing to admit.

https://www.deepmind.com/publications/a-generalist-agent

https://www.deepmind.com/blog/building-interactive-agents-in...

> he sees them the same way you see Indian/Japanese/Arab characters without being able to tell what they mean.

Lost me at Indian characters, only if he asked chatGPT Indian isn't a language.

Indian characters could be the superset of multiple character sets from the Indian subcontinent. There's nothing inherently wrong about it.

Agreed, i got that too. But i would like to people start noticing and not use general terms often. Nothing wrong with it.

Indians are people, same as Americans. Proper way to represent characters of languages originating from Indian subcontinent is Indic and/or dravidian languages. With Scandinavian language speaking countries we know who they are, maybe we use a similar context. :)

It's the perfect product for the average non-technical user.

> ChatGPT, summarize the article

No true Scotsman. Flamebait

I agree, but that doesn't make it any less amazing.

This article was written by ChatGPT.

Reply on news.ycombinator.com