The limitations of deep learning (2017) | Hacker News Reader

The limitations of deep learning (2017)

blog.keras.io

52 pointsandrelaszlo18 comments

IMHO, while anthropomorphizing AI is an error, it's only an error in the same sense that anthropomorphizing a cat or a dog or a cow is an error.

People tend to think MUCH too narrowly about the parameters of "intelligence" and/or "consciousness" and vastly overestimate the human perspective.

Are dogs and cats debating their equivalence of anthropomorphizing?

If anything I would say the opposite. We draw parallels quickly to other organisms and vastly underestimate the range of human cognition.

I don't mean to suggest that humans have little cognition, what I mean is that something can be intelligent and conscious even if it behaves in a decidedly non-human way.

Animals have their own kinds of intelligence, and they are all conscious, albeit with different kinds of higher functions than humans. But all the possible colours and textures of cognition tend to get lost when we reflect about the nature of AI.

We tend to boil it down (incorrectly IMO) to some variation of "it's not intelligent/conscious because it doesn't do X the way humans do".

Yeah the subtext people mean when they say not to "anthropomorphize" AI is "humans are the gold standard of intelligence, all of its limitations are in its failures to do things humans do easily or automatically, by anthropomorphizing it you are ignoring the AI's limitations".

There is this built in assumption that humans have a natural supremacy in the realm of "intuitive" information processing that is unquantifiable. As AI improves this somewhat supernatural view will dissipate.

In fact, in the future when people talk about not "anthropomorphizing" AI, it will be in the opposite sense, i.e. we will be warned not to assume that an AI has human flaws, because in reality it is much smarter than that.

Cats, dogs, cows, and other animals have better learning capabilities than deep learning systems. This is specifically because most DL systems are frozen, compressed snapshots of a static training distribution. The deployment distribution is not only dynamic, but malleable: output of the model changes the distribution of 'correct' outputs for most nontrivial tasks. Continuous unsupervised retraining and fine tuning is beyond the capabilities of deep learning.

> better

I don't mean to be pedantic or flippant, but it really depends how you measure. By any measure of correctness, a Casio calculator manufactured in the 1980s has superior intelligence to a human. I would just say "different", personally.

If anything, we should be more ready to anthropomorphize a LLM over a dog or cat.

It's being trained and evaluated on how effectively it can complete anthropomorphic data, and at this point has been shown to model world models abstractly present in the training data.

The trick is - as humans - avoiding binary thinking on the topic and recognizing there's almost certainly a spectrum going on, where certain aspects of anthropomorphizing like attributing a subjective continuous experience is silly, but other aspects like the model having some kind of abstracted modeling of emotional states, motivators, or concepts of self - unquestionably things modeled in the majority of social media data fed into the models - is seriously evaluated and considered.

We're too caught up in either or thinking, where no one wants to be seen as a tinfoil hat figure around "it's alive" so the 'safe' stance is denying any anthropomorphizing up until those people are caught by surprise by research around the performance benefits of emotional language or jailbreaking with appeals to empathy, at which point all too often there's an anchoring bias where they deny the new information as long as possible (another anthropomorphic feature that seems to have made its way into LLMs btw).

Less than 10% of what I see people discuss about models today seems particularly enlightened, and the leading minds like Hinton who straight up say you can't autocomplete well without underlying knowledge get dismissed by those eager to distance themselves from any accusation of being too quick to anthropomorphize the thing trained and evaluated on extending anthropomorphic data...

It's a ridiculous state of affairs.

"Say, for instance, that you could assemble a dataset of hundreds of thousands—even millions—of English language descriptions of the features of a software product, as written by a product manager, as well as the corresponding source code developed by a team of engineers to meet these requirements. Even with this data, you could not train a deep learning model to simply read a product description and generate the appropriate codebase."

I would love a centralized resource of all the predictions of what the technology would not be able to do in the future and then the dates at which those capabilities were first demonstrated on a giant timeline.

Maybe in the next few years I'll be able to ask a model to generate that for me. It will be pretty fun to look at.

You still can not "could not train a deep learning model to simply read a product description and generate the appropriate codebase", unless the product is very, very, very simple or trivial.

Pretty sure the text there didn't have an exception added in.

And the complexity threshold for the product where writing a description and getting working code is successful increases each year with no real reason to doubt that it will plateau, particularly as models hook into linters and static compilation in multiple rounds of correction over the next eighteen months.

"It won't ever do this thing" and "ok, it kind of does it in simple cases and may do it in increasingly complex cases as time goes on" are two very different statements with a gulf between them.

> "It won't ever do this thing" and "ok, it kind of does it in simple cases and may do it in increasingly complex cases as time goes on" are two very different statements with a gulf between them.

Until it's perfect, the amount of trust you can have in the latter case makes it equivalent to the former. There's no gulf really — you can't count on what the code "may do".

"It will never win a chess game"

"Ok, it can win a chess game, but if we don't actually keep track of the moves it is making we'll have no idea if it can win a chess game, and therefore it's the same thing as if it can't win a chess game."

You can always read code you didn't write, by the way.

He's not actually wrong yet though. You can now generate what appears to be a codebase but the chance of it functioning as specified as a whole is basically 0.

This is already over a year old at this point:

https://andrewmayne.com/2022/03/17/building-games-and-apps-e...

They're great demos, but the tasks are not really at the scale of larger projects and codebases.

Again - is the copy quoted saying that paragraphs of simple tasks and demos will be able to be coded and only larger complex software can't be written by description?

Because it really seems to be saying that no software at any level of complexity can be written from paragraph to working code by deep learning.

We should be careful not to move goalposts post facto.

And given the trend so far, it is likely that paragraphs of software with greater complexity will be able to be successfully coded as time goes on and models and infrastructure around those models improve.

So let's also be careful not to assume the status quo is the end of the process to be evaluated rather than a stepping stone between the beginning and the theoretical end.

Discussed at the time:

The Limitations of Deep Learning - https://news.ycombinator.com/item?id=14790251 - July 2017 (260 comments)

Very nice read from today’s perspective. Many points still hold, while a few have clearly progressed by leaps from back then. In hindsight language is a pretty nice generalizer as so much of it is available.

Reply on news.ycombinator.com