Hacker News Reader

Comment by ThomPete | Hacker News Reader

I am seriously interested in understanding why a lot of people have no issue accepting the idea of humans evolving from simple immaterial matter into biological beings and all the way up to our current reality but have a hard time believing that can be done via computers.

I would at least put it in the very likely box that computers can learn to be the same kind of pattern recognizing feedback loops that we are even without humans understanding the brain completely just as we became conscious and self aware without any "programmer".

"Computers" don't need to be like us to become intelligent, they don't actually need to reproduce the lungs and the intestine as we have it, they are in many ways free of those restrictions.

They might be "concerned" with very different things than we are and might even not really care about nature to survive. All they need is energy.

View on HN

I don’t think anyone here disputes the logical possibility, but rather that such systems are still far off, not as near as the singulariati would have us believe.

From the perspective of embedded cognitive systems, intelligence is not an abstract property but a faculty derived from an agent embedded in a world with an ability to act upon that world in specific ways. This is reiterated in cognitive linguistics by Lakoff’s metaphor work et al. I take this to mean that the question of AGI itself is missing the point. To wit all recent provoking demos of AI (Alpha Go, DOTA) are embedded within specific environments and actions of a model-specified agent. It is then extrapolated that AGI will appear automatically induced by a series of more and more competent AIs.

Incremental evolution as a means of indiction as we know from biology can take a long time, and the landscape is complex. Hence the view that it might be coming but not soon and not with a roadmap.

> not as near as the singulariati would have us believe.

I've seen the singularity described as "The Rapture for nerds", which is pretty apt. It's going to be amazing, the future is so bright, we won't have to worry about any problems plaguing mankind now, and it's just around the corner! Any day now! Aaaaaaany day now!

Exhausting. Maybe we'll get a future AI-run utopia a la The Culture. Maybe not.

> singulariati

I like this term. Going to use it any chance I get now.

Biology works slower generation wise than technology. If selfawarenes is based on emergent complexity in pattern recognizing feedback loops then technology can play through billions of scenarios in a short time. We dont need AGI for technology to be a problem. Technology at the intelligence level of a mouse with all its potential physical power through the systems it might have access to can be quite dangerous for humans.

> but have a hard time believing that can be done via computers.

Much of the skepticism isn't about whether this can be done in theory. It's about "how close" we are given the current state of the art (beating Go, chess, deep NN, etc).

One can be (I am) simultaneously blown away by these accomplishments and believe that we have hardly scratched the surface of AGI.

And while this isn't necessarily a deal breaker for AGI, it's not encouraging that we still understand close to nothing about consciousness.

> .. but have a hard time believing that can be done via computers.

> I don’t think anyone here disputes the logical possibility

> Much of the skepticism isn't about whether this can be done in theory

The math tells us it cannot be done with computers. Read Penrose.

Penroses opinion is not exactly mainstream science in this area.

> The math tells us it cannot be done with computers. Read Penrose.

I'm familiar with Penrose.

He's interesting, and I don't have a firm opinion on this (ie maybe it turns out it is impossible), but presenting the case as "the question of AGI has been settled by math" is really misleading. This is an area of open debate among philosophers, mathematicians, physicists, and evolutionary biologists.

Mainstream physicists, neuroscientists, and ML researchers are all more or less united in their view that Penrose is really overstepping the valid application of the arguments that he's using when he talks about this stuff. He really really wants quantum mechanics to be an important part of the intelligence/consciousness debates, so when he sees an indication that it could be relevant, he jumps to the conclusion that not only is it relevant, it is of paramount importance.

You seem to be missing a key ingredient: billions of years and endless branching iterations with the totality of all non-sequestered resources available. Even if you scale that to account for the more rapid iteration theoretically possible with machines, it doesn’t paint a rosy picture for any of our lifetimes.

Plus, in organisms the software and hardware are both subject to mutation, and exist in the context of a global system of competition for resources. That only tangentially resembles work done on AGI, and only in the software component. We can only design and build hardware so quickly, and that adds more time. I’m not hearing pushback against the possibilitY of AGI, just singularity “any day now” claims that seem mostly calibrated to sell books and seminars.

> I am seriously interested in understanding why a lot of people have no issue accepting the idea of humans evolving from simple immaterial matter into biological beings and all the way up to our current reality but have a hard time believing that can be done via computers.

And similarly, I’d love to understand why you’d believe in something for which there’s zero evidence and ask with incredulity why others don’t also believe it. Isn’t it reasonable to be skeptical until it actually happens? Isn’t it possible that we don’t understand what intelligence is, and that there could be an undiscovered proof that computers can never attain it? It seems too early to call.

Computers haven’t to date ever done anything on their own, what reason is there to think they ever will? We made them. Neural networks are nothing more than high dimensional least squares. Maybe humans are also least squares, but my money is on there being more to it than that.

> ...they don’t actually need to reproduce the lungs and the intestine...

Obviously, but the one thing that all intelligent life does is reproduce and contain a survival instinct. No computers have a survival instinct, and thus no reason to get smarter that we don’t imbue them with artificially.

> Computers haven’t to date ever done anything on their own, what reason is there to think they ever will?

Interestingly, people that spend a lot of time studying their own brain by meditating are usually very confident that what most people call "free will" does not exist - is merely an illusion.

Not to mention that physics tells us that we should be simul-able, after all physics knows only about deterministic and random processes.

There is a noticeable bias among computer scientists to believe in simplistic models of intelligence and minds. I agree with you, though, and I also sympathize to the ideas of Searle and Penrose.

>and that there could be an undiscovered proof that computers can never attain it?

It's already implemented in at least one physical system.

Would love to read more. Do you have a link to this project?

I meant that human brains exist.

Right, I realized I misread your comment. See my sibling reply just above. I'm not certain that the existence of human brains proves anything about what computers can do. While it seems logical, nobody has shown it, and nobody has established a definition for life or intelligence that proves it's computable, right?

Oh haha, duh, I misread your comment as talking about the proof, rather than life. I guess my response is then, if it’s just physics, why aren’t computers mating and reproducing and getting smarter on their own already?

Aren’t there some kinds of math that are proven to be unable to solve some problems while other kinds of math can? Trisecting an angle can’t be done with a ruler and compass, and a 5th order polynomial can’t be solved analytically. Both of these things can be done numerically.

Is it possible then that some physical systems can’t act in the same ways as other physical systems? A granite rock can never burn at the same temperature as a tree, but they’re both physical objects. Is it possible that binary logic on silicon doesn’t have the physical properties for intelligence that animal brains do?

Also ponderable - there aren't many other intelligent animals around, but that is not actually evidence that human intelligence is hard to achieve. It is evidence that it is hard to turn a small increase in sub-human intelligence into an evolutionary advantage.

There is solid evidence that teams of mathematicians, scientists and engineers will be able to catch up with what nature has wrought in the next few centuries, if not much faster (decades on the optimistic end). Intelligence could be one of the easiest part of biology to replicate, given how shonky human intelligence is when tested against objective standards.

I agree with your first paragraph, but your second one is definitely wrong. We aren't anywhere near even understanding "what nature has wrought" the possibility of "catching up" within the foreseeable future is not realistic.

I question what objective standards are here. Human intelligence sucks in terms of consistent long term memory, and possibly in terms of not being influenced by emotions/outside forces. It's pretty good at a number of things computers are terrible at however, including ability to generalize from very limited "training" examples, combining "models" i.e. general object recognition with physical movement/obstacle avoidance, lots of things having to do with natural language. What kinds of objective standards were you referring to?

For my part I have faith that computers are great at memorization, and will continue to improve on that front. However, I'm less convinced on their ability to "understand", which is admittedly poorly defined, but intuitive. It seems to me there's still a missing piece between machine learning (essentially all things that are called "AI" these days) and the kind of generalization that we expect out of even a human 1 year old or your average vertebrate.

AI agents need environments with a complexity similar to that of our own in order to understand, and a goal to optimise on (humans have 'survival' as the goal). The missing link is that intelligence is dependent on environment and AI agents don't have rich enough environments yet, or a long enough evolution. The level of understanding is related to the complexity of the environment.

But that can be fixed in simulation. That's why most RL research (RL being the closest branch to AGI) is centered on games. Games are simple environments we can provide to the AI agents today. In the future I don't see why a virtual environment could not be realistic, and AI agents able to 'understand'.

We are a part of nature too. So when humans create AI isn't it still just nature creating AI?

...is strong AI a more sophisticated invention than cells? Is AI more sophisticated than DNA? Is AI a more sophisticated invention than a habitable orb of magma orbiting a nuclear explosion in space? It's all crazy. It's all amazing. ...We are all lucky enough to be here for a little while to see whatever this is. We're a point on a stream.

But let's not forget evolution of the magnitude you speak of took hundreds of millions of years. I'm not sure why humans think we can beat that record.

I don't think humans can beat the record, but technology? That's another matter all together.

Again our own consciousness wasn't created it emerged which means that we are a product of this emergence.

Technology at this time is nothing but an extension of human will. There is no indication or path that it has yet left our grasp. Therefore we are the limiting factor. Just as nature or the divine might have been the will that allowed us to emerge.

but if it ever leaves our grasp, you won't notice when it outpaces us, it's going to be so quick.

Because they aren't starting from nothing? They're starting from humans -- the more we can usefully encode about our knowledge of ourselves, the more time we can skip in evolutionary effort. The mutation rate is also rapidly accelerated and although we drop the fidelity in simulation, for the most part we can run magnitudes faster than real-time (and certainly if a limiting factor was human decision making time).

Life evolves at linear rates, while computers at geometric.

Even more to the point the cycle time for human evolution is at a minimum 14 years (biological limit to reproduce) and around 25 years (today's societal constraints). Whereas cycle time for new computer generations is what 12-18 months?

So at best every 14 years we get a new 'human model' with some random variations. Meanwhile in that time frame our computing hardware is 2000x improved (hopefully improvements in software accelerate that improvement further).

Flying took billions of years to evolve, but we created machines that fly faster than the speed of sound...

> but have a hard time believing that can be done via computers

Remember that we are also constrained by our own intelligence. Computers might not be the problem.

You just described AI in the 90's: Artifical Life, subsumption architecture, evolutionary algorithms. Theoretically we should be able to evolve intelligence but the search space is impossibly high and we don't understand why life is evolvable. Even if we did evolve an AI through a simulation of evolution - there's a small chance we would understand it.

> I am seriously interested in understanding why a lot of people have no issue accepting the idea of humans evolving fromm immaterial matter into biological beings and all the way up to our current reality but have a hard time believing that can be done via computers.

It took nature what like 4 billion years to come up with the human brain in a single species... We including Kurweil think that this can be replicated in say 100, I mean not even that according to the singularity, 40 years. I seriously doubt it. Can the human brain eventually be surpassed, yes, but it's very likely that we will go extinct before then.

One reason is that technology has the ability to do trial and error through billions of generations in a very short time.

Yes, but each "trial" exists in a simulation that is many orders of magnitude less complex than the one evolution used over billions of years. The result is that the knowledge learned from each trial is many orders of magnitude less useful.

It also took billions of years to evolve heavier than air flight, but it took technology much less time than that.

How long did it take evolution to build a structure taller than 100 m?

Technology just moves on different timescales, with different methods and different objectives. Evolution is slow and dumb, and not goal oriented.

In some domains, technology might never catch up to biology. In others we've beaten it soundly.

Planes don't fly like birds. We have ornithopter which are crude imitations of bird flight. Very crude. Similarly our AI are a very crude imitations of the real thing and I believe will continue to be for some significant time. At least a few hundred years for a somewhat plausible imitation of an intelligent machine. I believe extinction of the race within the next 10,000 years is incredibly likely. I also don't think the intelligent machines that we build will last very long because they will be difficult and expensive to build and because of the incredibly small size of their tech, say 1 nanometer or less, will be prone to various kinds of failure and also require rare materials that are in limited supply and can't be recycled so they are difficult to obtain for both us and for the AI to maintain its own supply chain without us. So our bots will go extinct soon after us.

Consciousness transfer is probably 10s of thousands if not millions of years away.

> all they need is energy

A comment above about being as intelligent as a mouse seems to touch on this problem domain rather elegantly : "navigate the forest floor looking for food and avoid predators".

> the idea of humans evolving from simple immaterial matter into biological beings

We have no evidence that life evolved from inorganic matter (i.e. that life 'began' at some point). That's a widely made assumption but it remains only that. The alternative is that life has always existed (and continually evolved).

Reply on news.ycombinator.com