Yann LeCun: Human-level artificial intelligence is going to take a long time

AI is quite troublesome for privacy though. How much privacy humans need is a question we'll probably have answered the hard way.

Both the USA and Nazi Germany benefited massively from have a civilian industrial base that was complementary to military production.

Would you shut down the powerhouse of our economy -- travel, transportation, energy -- for something hypothetical that hasn't even happened and doesn't appear to be close to happening?

I'm pro-clean energy, but you can't do without fossil fuels. Not if you want society to keep climbing up and up and up.

Evolution also crossed the flight barrier by banging molecules together. I don't think banging molecules together without having an understanding of physics and the forces involved would have been a viable means for us to get to flight.

"Machine learning isn't the route to AI" is something you could more reasonably argue, but that's a drastically narrower claim.

NP Hardness is a statement about the asymptotic difficulty of solving a problem at ever larger scales. It says ‘if you have a way to solve this problem at size n, that way will scale worse than polynomially when you try to apply it to a problem at size 2n’.

Which might place limits on the practicable scale of how big an n your approach gets to work for. But if your approach works practically for a big enough n to make AGI then your approach works - NP Hardness doesn’t matter.

And since we know that finite mass lumps of finite numbers of gray cells are capable of GI, we have a reasonable expectation that there is some n for which AGI might be possible.

A "machine" is what, exactly? Do we take it to be an abstraction? Or is it an electrical field oscillating over silicon? Either way, you're in trouble. Abstractions have no physical properties, and electrified sand seems hardly to possess any interesting properties.

The ability for animals to adapt to their environments, by growing into them, by establishing plastic causal connections in their very bodies, grown by their environemtns... able to almost instantly move from protein expression in 1trn 1-bn-yr cellular supercomputers in each of our bodies to macro sensory-motor representation --- and back again

Is this ineffable?

Or is this extremely effable. Is rather, not the superstitious view that "everything is anything" ?

All extant, knowing, studied intelligent systems have organic properties; and radically so. Insofar as this is "ineffable" you should take that up with the animal kingdom.

I find the contrary supersitious, magical, religious, ineffable... that mere abstract patterns in arbiatrily chosen aspects of our bodies are necessary and sufficient conditions for anything at all. This would be the only case in all science. The only physical property instantiated by mere arrangement at any level. Upload our consciousness? Make it out of wood why not!

Nonesense. When I am hungry, I dream of food, when I dream of food I plan to get some, and I am angry without it. This will not be made out of sand.

The former can be seen as a religious belief, but the latter is simply saying that we do not understand anywhere near enough about intelligence to develop AGI.

You point to the unexpected leap in capability we've had recently, but fundamentally it isn't that different from what we have had for decades. The same fundamental unsolved limitations exist, such as of being unable to learn the way we do (eg if a child is writing a specific letter wrong, we correct that specific mistake, but that doesn't overwrite the knowledge of how to write the other letters).

As a lecture I watched a while ago had put it, we've gotten pretty good at making the icing of the AGI cake recently, but we still have no idea what the cake is supposed to even look/taste like, so we barely know what ingredients we might need to make it let alone the actual steps involved.

> As a counterpoint, I feel like the belief that we can't get to AGI comes off as religion. It presupposes an ineffable quality that we posses that machines cannot.

That's missing the point: the argument isn't that it's theoretically impossible to build an AGI, just that humans are incapable of it.

Here's another point for the belief in AGI being a religion: it's basically a sect of the larger religion of technological progress, which (likely falsely) assumes that technology will continue to "advance" at approximately current rates until we live in a world out of a sci-fi paperback. A lot of people believe that, but frequently resort to a motte-and-bailey fallacy when challenged.

> It's always hard to predict the rate of progress, Most of the current optimism comes from how radically wrong predictions were for the capabilities of AI today. 10 years ago a lot of people would have put current AI capability as arriving well after 2050. The jump in progress may not be sustained, but it definitely places doubt on people confidently predicting slow progress.

It's worth noting that "the current optimism" is not without precedent. IIRC, there was a big boom in AI in the 70s/80s. SHRDLU was pretty impressive. But then then the promising ideas were found to be dead ends and there was a long winter.

An example of this is electricity. The battery (1800), electric motor (1821) and telegraph (1832) were all invented before the discovery of the electron (1897).

The inherent capacity for true self-directed learning may well be there, but it isn't hit yet in what we see.

GPTs are still neat.

Among the small but growing number of modern idealists (the philosophy that consciousness precedes matter), a great deal are actually computer scientists who originally set out to work on AGI from a materialistic frame of reference, but bumped into the conscious machine problem. Once they more closely explored what the cognitive fields working to find answers to this problem are saying, they realized that most scientists approaching consciousness as an emergent property of matter will probably never get anywhere with this.

Why is "the hard problem of consciousness" inextricable from human-level AGI? Because intelligence is only one component of our mentation. There is a back and forth between processing and experience that make up the whole thing. A machine can have or even exceed human processing power, but it will not experience the simplest of colors. Its approach to "creating" can be random hit-or-misses, or formulaically from existing input. But without subjective experience it cannot be as intently creative as an artist trying to come up with a new genre, or as funny as a comedian that relies on the shared subtleties of feelings and emotions (conscious experience) to make original work that other humans can resonate with. It cannot have the intuition of a gifted chef who suspects that two seldom combined spices might be an unexpected but resounding success in her new dessert.

At some point, it seems important to stop trying to understand the world through ideas and concepts and just experience it.

I really think there are more religious like beliefs on the pro AGI camp than the other way around, think about what AI promises to deliver by "visionaries" like Hinton and Kurzweil:

* Immortality.

* Resurrections.

* Nirvana / Heaven / Utopia.

For all we know, we create an AGI who becomes amused with fucking itself, or develops new games it finds interesting and spends all eternity playing them. Humans have a belief that we're going to create "god" as we think of "him". All seeing, all knowing, all powerful. It does get a bit ridiculous at times.

I sympathize a lot wit the parents view personally.

Physics cannot formulate descriptions of climates, or even, many cases of turbulence, or 4 bodies in a gravitational field.

Leaving "physics" is trivial, it occurs whenever you take any of the current toy models and add one layer of reality to them.

The whole edifice of formal science is a little like children's block toys.

"The Truth About the [Not So] Universal Approximation Theorem" - https://www.lifeiscomputation.com/the-truth-about-the-not-so...

I think you're using a completely different definition from everyone else.

That's clever but it doesn't help to start redefining terms that are the basis for a discussion with others who use that term in a different and more generally accepted way. General doesn't mean 'generally trained' it specifically means that it does not require training for a particular task in order to figure it out. That implies that it may not require training at all and that if it is trained the training isn't necessarily general but that the AI can extract useful patterns from training on completely different subjects and apply them to the problem at hand.

This is subtly but crucially importantly different from 'AI with very general training'.

After combining enough specialized modules together with a management layer, we will essentially have AGI. Of course it won't be human, because human intelligence is tightly coupled to human perception.

The point is why should we take that as true when people didn’t have a clue LLMs would take us this far, until they did.

Btw there are lots of ideas on how to deal with the data issue. Multiple directions in research are promising to work around that.

It’s the convergence of end of ad economy/attention supply growth and Microsoft’s money forcing AI as the next growth narrative that’s forcing google to join the party - at massive cost to them.

That's made them effectively immune to copyright enforcement so far; I'd go as far as saying the main function of llm/diffusion models is obfuscation of copyright breaches.

Turing was right, but he believed in human intelligence, so thinking linear.

Unfortunately, neural networks appear harder then sought, because Forward-Forward learning with which working human brain, is now considered too computationally hard to be used practically.

Reverse-Propagation neural networks are much easier computationally to achieve, but are magnitudes less dense than FF, that's why we have so much delay - just need thousand times more performance, this is about 30 additional years.

PS I tried Llama 30b on my computer (honesty, tried all Lllama models could fit in 64G RAM, from 13B and more). It is not sprinter fast, but I'm very impressed, on how deep it could think. I must admit, I don't want to talk to many humans anymore, as It looks smarter.

One side project I'd love to work on one day would be gathering a dataset of Factorio maps designed to help train a floor planner. The requirements aren't the same as for making a PCB, but they're similar-ish.

That remains to be seen. You wouldn't say a human lacked human-level intelligence just because they had only read every book in The Pile, rather than every book in the Library of Congress.

Multimodal (image/video, audio) is barely being scratched. You likely could 100x training compute and not run out of video content.

Not to mention synthetic data from simulators, compilers, and other sources of infinite ground truth data.

If you listen to the big labs (OpenAI, FAIR, etc) they have started to say they are not worried about data anymore.

Quite obviously, at some point you put the agents out into the real world and they can collect new training data there too; this is what all the ChatGPT plugins are allowing OpenAI to do. The plugins that give good responses which are valuable will be disproportionately popular and therefore overrepresented in the new training set of user interactions.

At least for the time being there is no magic workaround for the amount of FLOPS and memory bandwidth needed. When you look at neural network architectures they tend to be massively parallel, and building out at that scale in hardware is going to cost a lot.

And, when the hardware does show up, at least with the scaling laws we're seeing at this time, the groups that have massive amounts of this lower powered/accellerated hardware are going to be ahead of those without budgets.

I don’t see why AGI would follow a different course, one where we are “just some breakthrough away”. Especially not given the current state of the field.

Personally I’d wager neuromorphic computing is the prime candidate to yield AGI.

"The first known working Newcomen steam engine is built by Thomas Newcomen with John Calley, to pump water out of mines in the Black Country of England, the first device to make practical use of the power of steam to produce mechanical work."

JIRA trying to solve the problem by lowering the "human level".

Also, Pinker isn't a scientist of any sort, he's an accredited and institutionalized evangelist. You guys wouldn't take opinions of someone going to conferences and trying to convert you to use Go lang instead of something else as seriously as those of Guido Rossum would you?

His whole thesis requires an idea of linear progress which is highly contested, and the metrics of happiness he uses also can be easily "Goodhart's Law"ed by him to show whatever he thinks is happening.

I'm not even getting to his Epstein stuff, which some might cry foul about, but I grew up around regular people and not in Silicon Valley, so a liar and a cheat in one field is probably a liar and a cheat in another to me, sorry.

And what are some examples of those things? Right now, AI, i.e., machine learning, is just being used to steal people's data and work only to try and sell them more things. That's 99% of the use cases out there right now.

More like we aren't making efficient use out of it. Many people are just trying make through the rat race or choosing to use their intelligence in games instead of working on interesting problems.

Enough for what ?

We don't have enough of what for what? And what do we even need it for?

We have 8 billion people on Earth, and we're running out of resources to handle all of them, including emotional resources. If we can't handle that, how are we going to handle billions of artificial entities with human-level intelligence? They will require oodles of power and have their own emotional needs.

All of this is chasing a poisoned carrot. And currently, all that we've gotten out of artificial intelligence is more targeted ads.

Here's also where I see it ending. It will need energy--likely a LOT, paid by someone, to do this. Who is going to pay that bill for it to maybe, maybe not, come up with something useful, likely mixed with mostly noise and distraction, over undefined timescales, of largely non-measurable value, when there's far greater value, less cost, less risk, in simply training it deterministically?

What people call "agi" but fail to accurately describe is an actual mind, with its own ultimate goals and ambitions. You can ask it to complete a task, and it may allocate some time for you. You won't own it. It will exist, grow and perform independently.