Comment by mjburgess | Hacker News Reader

Comment by mjburgess | Hacker News Reader

Parent Full thread

Representations in the animal sense can be observed as that which is in common across inferential contexts inn which animals are confronted by the need to use them:

My model of "pen" can be observed in action when: I write with a pen; I'm asked to pass a pen; when I'm asked "what color pen do you like to write with?" and so on.

What is this structure "pen"? Is it a latent pattern within my experience? No! It is a means of regulating my body, imagination, emotion, etc. "The pen is mightier than the sword!" is something I can agree, or disagree with; I have a feeling about it. And in inferential contexts which require me to feel, I feel.

This is what animals acquire when they understand the world. This is what everyone believes ML is uncovering, whether they realise it or not. We just dont have these ML-like templates, so we don't realise how thin and brittle they are.

When ML is run on text, is detects character patterns which form "character templates" that other text can be compared to. I cannot ask, sincerely, what the machine thinks about whether "the pen is mightier than the sword" -- because it has no attitude towards that proposition.

The character structure of the phrase "the pen is mightier than the sword" relative to a body of text, has nothing to do with any concepts -- that structure has nothing to do with the inferential contexts in which those concepts will become useful.

Rather, this thin character-space text structure can be repeated to people in the form of an illusion: we the dumb apes who have the concepts are fooled into thinking that some machine assembling of these words had something to say.

This is a very cheap trick, and fails pretty trivially. The relevant structure to be learnt, the representation, has nothing to do with sequences of characters.

Though the industry likes to talk about "latent structure" in this data, it isnt latent. It is right there on the surface, indeed: it just is that surface.

When the machine is challenged to do anyything other than report this average surface structure, it fails. ML systems dont work. They improve ROI on the margins, but automating basically broken inferential rules at scale -- it is the automation here, and speed that is useful. Not the actual quality of inference.

"They improve ROI on the margins, but automating basically broken inferential rules at scale -- it is the automation here, and speed that is useful. Not the actual quality of inference."

This is trivially incorrect, and again shows a lack of curiosity. A couple obvious counterexamples are AlphaGo (and descendants) and AlphaFold, both of which have quality of inference far out stripping previous efforts. In AlphaFold's case the results are pushing forward a whole field of science. Plenty of systems at this point in history outperform human experts, and not just on speed metrics.

'"The pen is mightier than the sword!" is something I can agree, or disagree with; I have a feeling about it. And in inferential contexts which require me to feel, I feel.'

What are these feelings? Are they necessary for learning? Will airplanes never fly until they learn to flap their wings like birds?

'we the dumb apes who have the concepts are fooled into thinking that some machine assembling of these words had something to say.'

Still arguing with straw men here...

'The relevant structure to be learnt, the representation, has nothing to do with sequences of characters.'

Of course it does. For BERT, the output is a conditional probability of words based on surrounding context.

OP makes great points that are not being addressed here. If you believe that animal intelligence is reducible to abstract symbol shuffling then it should be possible to extend the existing approaches to actual robotic automatons but it is clear that these models are unable to deal with the dynamic structure of the world and there is no reasonable avenue to actually make that happen. So either intelligence is reducible to abstract symbol shuffling or there is a semantic fib happening here that substitutes "abstract symbol shuffling" for "intelligence". The fact that we imbue these abstract symbols with meaning is where the intelligence comes from, i.e. people attribute intelligent behavior to these systems because we have learned to attribute intelligent behavior to the production of abstract symbols.

It's clear to me that current mathematics is insufficient for creating intelligence (let alone general intelligence). If you know of anyone that makes a coherent case for the mathematical basis of intelligence then I would like to see those references.

The OP makes overly broad arguments about AI that get details the details wrong and have obvious counterexamples in current ML techniques and results.

On the ML side, there's been steady progress with an accumulation of very visible wins, accompanied by nearly-invisible adoption of techniques in day-to-day usage. (eg, keyboard text completion and text to speech systems.)

Much of this progress has been in specific subdomains, as combining domains requires massively more effort and investment. However, it's starting to happen; CLIP methods create joint embedding spaces for multiple modalities, and have led to the zero-shot learning we see exhibited in DALL-E. This is stuff that simply wasn't possible five years ago, building on sub-domain tools which have greatly increased in their quality such as BERT.

These joint embedding spaces will get better, encompass more modalities, and fuel further results. For example, as joint text-video embeddings become more powerful, we'll have embeddings which jointly encode text and physics.

As a result, I would be extremely hesitant to write off robotics applications. There's no big-headlines breakthrough on ML robotics /right now/, but we also didn't have a clear road to zero-shot image generation from text inputs a couple years ago. Notice that many of the existing 'big' results were obviously impossible until they very much were not.

Finally, I don't believe that we have a good enough definition of 'intelligence' to say with any certainty whether current mathematics is sufficient or not. Don't underestimate the potential for simple components to produce complex behaviors, though. But also, don't expect Pinnochio; intelligence may be broader than the human implementation.

OpenAI shuttered their robotics division because they did not see any viable path to commercial applications so they pivoted to generating pixel art. Similarly, DeepMind has not been making any claims of achieving AGI because they're smart enough to realize that statistical modeling of physical systems is a very small subset of what counts as intelligence.

I'm not dismissive of the progress in the field. What I find confusing is why so many people are convinced that what we are seeing with these abstract symbol shuffling systems is intelligence. All it does is confuse the average person about what these tools are capable of because at the moment they are only capable of amplifying biases in existing data sets. No statistical model can escape this trap and at the moment we essentially have automated bias amplifiers that are being sold as some kind of revolution in designing intelligent systems.

Hardware is expensive to iterate on. ML research is already expensive, without worrying about hardware. I expect we'll see plenty of additional attempts in robotics, regardless of what makes economic sense for OpenAI in the short run.

"No statistical model can escape this trap"

Your claim here is that intelligence requires innovation?

AlphaGo certainly went beyond the bounds of the existing training data. Likewise, zero-shot learning (as we see in Dall-e 2) demonstrates the ability to combine concepts combinatorially, rather than drawing from raw prior observation.

I still wouldn't call this intelligence, but it's yet another indication of how the goalposts move in the conversation. (Never mind that we typically at this point ask to satisfy indicators which most humans could not satisfy...)

For just about any simple indicator of intelligence there's been a concerted effort to make a neutral network with that property. And most of them have had a degree of success, moreso over time. The 'confusion' comes because these simple indicators have repeatedly been set and overcome.

To me, these arguments are vaguely reminiscent of the philosophical arguments from the 80s and 90s. I also remember some people using Go as an example of a problem to which ML approaches won't work. We've gone from people giving up on computers solving Go, to human Go masters retiring because AlphaGo is impossible to beat.

I mean, AlphaZero is trained solely on self-play. It is not even given the rules of the game, it exists in the world where it is rewarded or punished by the 'laws of physics' of the Go board the way we exist in a environment with physical rules that constrain and reward or punish our biology.

To say that AlphaZero is just data compression of the inputs seems hand wavy. It is data compression only in the sense that phenomena from the world is stream of data, and humans developing representations of that data (eg laws of physics) around that data are a compression of it.

But AlphaZero wasn't given a huge feed of pre-played world data. Rather, it interacted with, and poked around in a simulated environment, until it was able to make good predictions on how its interactions would turn out. I learn that dropping a ball falls to the ground, and so I can make a prediction of what happens if I drop a ball. How is AlphaZero predicting the outcome of moves purely from self-play just another kNN? If so, why isn't our brain's learning just a kNN then?

My claim is that intelligence is more than just statistical associations and abstract symbol shuffling. It's impressive what large statistical models can do but they still can not solve sudoku so something is clearly missing here because neural networks do not have feedback loops and backtracking. It's like saying all we need to do is continue building bigger and bigger abaci and stacking them in just the right way as to emulate the statistical properties of the real world. Dall-e is dazzling but it is still a statistical model with no symbolic understanding (it's still a giant abacus). It's obvious that people have symbolic understanding (e.g. written language, mathematics, solving sudoku, writing code/software, etc.). So if people are the benchmark of intelligence (dubious but let's assume for the sake of argument) then at what point do you suppose there will be statistical models with symbolic understanding? Furthermore, what reason is there to believe that larger and larger statistical models are going to get us closer to non-human intelligent systems that do more than generate stimuli adapted to our senses?

There is also a meta-problem that no one seems to address when discussing AI. All the systems we have built rely on compositional symbolic systems (mathematics) for expressing statistical associations and human interpretation of their inputs/outputs. Clearly there is something people can do that no existing AI system can which is to generate a symbolic description of statistical models that can be adapted to various data sets.

I could say more here but none of what I'm saying is anything new. Others much more capable of describing the issues and shortcoming of the existing approaches to AI have written books exploring the issues in much more detail, e.g. Gary Marcus, Melanie Mitchell, Douglas Hofstadter, Gian-Carlo Rota, etc.

"It's impressive what large statistical models can do but they still can not solve sudoku so something is clearly missing here because neural networks do not have feedback loops and backtracking."

Again, the things you ask for exist. Recurrent networks and reinforcement learning both have feedback loops. (And there's a reasonable argument that residual networks can be interpreted as 'unrolled' recurrent networks.)

Here's a completely random paper on reinforcement learning for Sudoku with non-zero win rates (and a few other games): https://arxiv.org/abs/2102.06019

I'm not sure anyone's bothered to take a real crack at Sudoku specifically. It's another example of a weak indicator, though: someone will happily solve it if you're willing to call it the bar for intelligence. Given where we're at on game-playing generally, it seems very doable with current technology.

"at what point do you suppose there will be statistical models with symbolic understanding?"

Understanding again has no real definition, so this is open to endless argument. I think it's fair to say that DALL-E understands what an astronaut looks like, though.

Ok, thanks for the references.

Reply on news.ycombinator.com