Hacker News Reader

Comment by bjourne | Hacker News Reader

So the idea is that statistical language modelling is not enough. You need a model based on logic too for "real" artificial intelligence. I wonder what the evidence for this claim is? Because the inferences and reasoning GPT3 is already capable of is incredible and beats most expert systems that I know of. And GPT4 is around the corner, Stable Diffusion was published like only a few months ago. I don't see why not more compute, more training data, and better network architectures couldn't lead to leaps and bounds of model improvements. At least for a few more years.

View on HN

> Because the inferences and reasoning GPT3 is already capable of is incredible and beats most expert systems that I know of.

This is patently FALSE. You can, however, re-run a given prompt 10+ times, tweaking and nudging it into the direction you know you want, until it produces a seemingly miraculously deep result (by pure chance).

Rinse and repeat a dozen times and you have enough material for a twitter thread or medium post fawning over gpt-3.

I don't necessarily doubt you but can you give me an example of an expert system that is more capable ?

GPT3 can’t perform algebra over all 32 bit numbers. A trivial Python script can.

It behaves more like your nephew than a computer in that case. Interesting that this is often the example given for why computers are bad at certain tasks, and humans are good at others.

It is quite incredible that nothing changed about the architecture in gpt-2 vs gpt-3 (just way more connections), yet it aquired fundamentally new behavior - that if performing arithmetic calculation - despite not having large amounts of training data on the subject. I think this is the type of phenomenon that shows we are quite poor at estimating what these systems will be capable of when scaling up. So acting as if we're sure it won't lead to improvements in AI is as idiotic as claiming that it will. There are far too many people on hacker news that follow this fad of being dismissive of AI, because they make the common mistake of equating cynicism with intelligence.

It’s smoke and mirrors trying to fool you into thinking it’s generating intelligent text. In some applications e.g., a chatbot, that’s appropriate. But it’s really no comparison to an expert system for most applications, where you know exactly the right and wrong solutions. Not adding numbers correctly with the huge budget GPT3 has for training and inference is a poignant case of that fact. A linear layer taking in x and y will learn x+y just by setting the weights to 1.0, so it’s not even a hard problem for neural nets, just in the particular tokenization and architecture used for GPT models.

> Stable Diffusion was published like only a few months ago

Honest question: what's "intelligence"-like about Stable Diffusion?

Because being able to draw "a painting of joe biden as king kong on top of a skyscraper in the style of monet" was something that until very recently were thought of as requiring intelligence. Of course, now it is not so impressive anymore because it is all mathematics and digital logic. But that is the problem with defining artificial intelligence. Any time a task is implemented on a computer you can point to that implementation as evidence that the task didn't require intelligence after all. Many decades ago researchers thought that playing chess on a high level required intelligence, then go, then poker, then composing music, then driving a car, etc... Nowadays researchers are more cautious and don't state that "solving task X implies intelligence". Thus it becomes a moving target and a computer can never prove itself intelligent.

Turing test.

The Turing test in its original formulation has already been soundly defeated. People now hedge their bets and require that an AI must fool leading AI researchers to pass the test. But the original test supposed that the interrogator was "average" and fooling 99.99% of the world's population must be good enough. Either way, as LaMDA demonstrates, it is only a matter of time before even the strongest imaginable version of the Turing is also defeated.

Was a publicly run Turing test ever defeated? I could not find any record of such.

Yes, it was. Your weak google skills notwithstanding.

The bear in the movie Annihilation passed the Turing test, but it didn't seem to have much intelligence

By some metrics, intelligence is self evident. Especially in the context when you mean approximately the same thing between intelligence/conscious.

There is some intangible property I observe when I look at a human and determine they are conscious. There is some intangible property I observe when I look at a dog and determine it is conscious. There is some intangible property I observe when I look at stable diffusion and determine it is conscious.

Some attempts to explain this intangible property have been made. Almost all of the time disagreements in these explanations boil down to semantics. Yes, I consider the ability so solve problems a demonstration of intelligence. Yes, I consider to Stable Diffusion to be solving problems in this way. Also yes, I consider a hard-coded process to be behaving in a similar way.

At the end of the day we seem to define consciousness as something that makes us sufficiently sad when we hurt it.

Reply on news.ycombinator.com