Gödel's Incompleteness Theorem and Its Implications for Artificial Intelligence | Hacker News Reader

Gödel's Incompleteness Theorem and Its Implications for Artificial Intelligence

sabinasz.net

43 pointsbehnamoh28 comments

Human minds will go crazy when deprived of external stimulus.

This is both their greatest limitation and their greatest advantage.

For this, I will refer back to Turing original essay about M-Machines. These are the universally known Turing machines.

But these were not the only machines Turing theorized in his seminal essay. There are also O-Machines, who take the result of the computations, not from the intrinsically closed mathematical world they inhabit, but from some external Oracle. Hence, O-Machines.

We are Turing O-Machines. The Oracle is out there. It's in all the external stimulus we take, in the world that's external to our minds, and in how we interact with this world.

It's the same for any artificial intelligence. Gödel's incompleteness applies to M-machines only. These are closed systems. Any mind with external interaction is not an M-machine, sometimes it takes answers from outside, and is, for a brief moment, an O-Machine.

Is consciousness simply the art of being unpredictable?

Is HN a type of Marvin Minsky's society of mind with more consciousness than a pet rock?

human minds will immediately cease to exist when deprived of external stimuli. We are not O-machines and our minds cannot be separated from the environoment

It's funny how the mind is mathematical to mathematicians, a compartmentalized system to psychologists, an engineering system to neuroscientists, a small representation of God to religious people, an emotional system to artists, etc... like the parable of the blind men and the elephant.

I sometimes wonder if our minds can create the reality we wish to some extent so we all live in our own universes.

I studied math in undergrad with a philosophy minor. To me, it was pretty clear that math absolutely was 'alien'/foreign to the human brain. It's clearly very useful, but human brains are not inherently capable of grasping it. (But they can learn remarkably well...) Notably, Godel died by malnutrition after convincing himself he was being poisoned or misled about reality.

Psychologists seem like semi-empirical philosophers. Philosophers seemed, usually, to be conveying their world views in their writings. They'd claim universality, sometimes. Other times, the arguments seemed almost pathologically designed to shirk criticism in the most trivial, petty ways (thinking of you, solipsism) - and we should really all just move on from them to more meaningful discourse.

[Engineers should be trusted with absolutely nothing but the exact problem space their disciplines have proven reliable. They're like children, at worst.]

Nonetheless, I suspect Godel's incompleteness theorem may be a sort of logical breakthrough the ramifications from which are still not properly explored.

> To me, it was pretty clear that math absolutely was 'alien'/foreign to the human brain. It's clearly very useful, but human brains are not inherently capable of grasping it.

That pretty strongly disagrees with a few experts on the matter.

https://en.wikipedia.org/wiki/Where_Mathematics_Comes_From

https://en.wikipedia.org/wiki/Mathematics,_Form_and_Function

Also, I'm not sire how Godel's death is relevant.

From your link: "It's difficult for me to conceive of a metaphor for a real number raised to a complex power, but if there is one, I'd sure like to see it." — Joseph Auslander"

https://www.math.toronto.edu/mathnet/questionCorner/complexe...

We do grasp mathematics, some of us anyway, so clearly that’s false but I doubt it was meant literally. The linked articles are interesting, thanks, they seem to show the cognitive mechanisms we use to engage in mathematical thinking. I’d rather say that understanding or performing mathematics is, how to say this, cognitively expensive for human minds. It takes an awful lot of effort, because while we have cognitive mechanisms suitable to the task, there not very efficient for that purpose.

> Also, I'm not sure how Godel's death is relevant.

Godel did a lot of seminal work on how to analyse statements and beliefs and how to reason about their truth or falsehood, yet died as a direct result of holding a (presumably) false belief.

The books those articles summarize are very good, although I haven’t completed either.

I’m familiar with Godel, but he was mentally ill. He had several phobias and mental disorders, and at the end, his wife became severely ill and was no longer able to care for and feed him, so he starved himself to death.

Mental illness and the fragility of the human mind and psyche have little to do with mathematics being first and foremost a human creation and endeavor.

“Why was man created alone? Is it not true that the creator could have created the whole of humanity? But man was created alone to teach you that whoever kills one life kills the world entire, and whoever saves one life saves the world entire.” —paraphrased from the Talmud

Thanks for this

There are no implications from Gödel's theorem that apply to AI that do not also apply to people.

I think that is the point of Chalmer's response to Penrose, referred to the article. [0]

[0] https://consc.net/papers/penrose.html

Yes, the article is a form of clickbait, posing a bad premise and then citing a debunking.

This is something of an anecdatum, but of the formally trained philosophers and logicians I know, none regard arguments of the sort Lucas/Penrose advance as viable.

Can any of them programme computers?

Some of them, but they don’t do it much. But they have quite good maths/TCS knowledge overall (one did part III maths at Cambridge, one seems to know all of Barendegt’s lambda calc book…)

They present a fairly convincing argument that human intelligence is impossible.

Gold

Penrose would say that logicians can’t compute consciousness, so no wonder they don’t agree with his argument; computers (logicians) literally can’t! That’s Penrose’s very argument, which so few seem to get.

I see plenty of evidence that they’re right.

Tangentially related, but I have often thought that we need to update how we explain some of these classic, 20th century computer science ideas (Gödel's incompleteness theorems, the halting problem, the P vs NP problem, etc) to 21st century audiences. These ideas are often incorrectly summarized as statements of the form "computers will never be able to do ___________", which is no longer a great explanation in a century where computers are already starting to do ___________ and widely projected to do far crazier things, for any choice of ___________ you want.

What progress are computers making on being able to tell if an arbitrary program halts?

The halting problem is only one example of what we would today call an RE-complete problem. Another example, which is equivalent in strength to the halting problem, is mathematical theorem proving, which is precisely the topic of this very article (as this is one way to interpret Gödel's theorems).

Any algorithm powerful enough to generate a proof or refutation of an arbitrary sentence in first-order logic is equivalent in strength to one which can solve the halting problem. This general framework of thinking has often led people to declare that computers will never be able to perform automated theorem proving of arbitrary theorems, for this very reason.

This very narrow analysis doesn't square well with the "real world," particularly in the 21st century, where it is widely projected that AI will eventually get, let's call it, "very good" at this kind of thing, with much fanfare about how it will design the next generation of itself and possibly turn planet Earth into grey goo in the process.

These two perspectives are not really in conflict, because when we talk about an AI that is "very good" at doing math - one exponentially better than a human, which is capable of solving any problem a comparatively puny human mind is capable of coming up with, or perhaps using stochastic methods with a small but nonzero failure rate, and so on - we are talking about an infinitely lower set of goalposts than we have when we talk about the traditional halting problem. To qualify as solving that, an algorithm must be "infinitely good" at this task - it must never fail, even if the probability of doing so is exponentially small, and its failure mode can never be an infinite loop, etc - all things which we do not care about in a realistic machine learning sense.

These things are really very technical statements that need to be interpreted in a very precise and careful way - they are very powerful ideas that shaped 20th century computing, but when we carelessly use them to make claims like "computers will never be able to do _______," we do them a disservice.

How much of intelligence relies on axiomatic logic systems(natural or artificial)? Neural networks are more like a traversal of probability fields that are neither purely internally consistent or purely complete.

The connection between Gödel and AI made more sense back in the 60's, when people thought AI would look like a formal system proving theorems. Nowadays modern AI techniques are more like a black box of matrix math streaming out sentences, images, and other things without a crisp, logical rationale for it.

Modern AI does not work like it is "proving" things. So, whatever limitations it has, Gödel's theorems aren't too relevant.

An informative overview which is not referenced in the article is Torkel Franzén's Gödel's Theorem: An Incomplete Guide to Its Use and Abuse. (https://www.goodreads.com/book/show/12830.G_del_s_Theorem).

Reply on news.ycombinator.com