Hacker News Reader

I also was of the opinion that there was the possibility to reach general AI in the future, but now I will take time to read this paper. I'm not skeptical about it because we know your assumption number 3 to be false: there are mathematical functions that aren't computable. For a proof you can look here [1] or google it. [1] https://www.hse.ru/mirror/pubs/share/198271819

Thanks. My assumptions are imprecise, because as I mentioned I am no academic. So I appreciate your correction. To make the post stronger, replace assumption 3 with "All of the mathematical rules which underlie physics are computable." IMO, this doesn't materially affect the plausibility of these assumptions.

>> "All of the mathematical rules which underlie physics are computable."

I really have to ask- what are those mathematical rules that underlie physics and how do you know that they are computable?

All of chemistry and physics as far as we're aware are described by mathematical rules that can be computed. Things like the strong and weak forces, gravity, etc. all have mathematical descriptions. There may be other, non-computable aspects to physics, but I am not aware of them if so.

>> There may be other, non-computable aspects to physics, but I am not aware of them if so.

In that case, what do you base your assumption (3) on?

Let's consolidate this discussion into the other thread, since it seems like you're offering the same objections here and there. :)

Wait, you forgot one interesting question. This paper claims that a Turing machine can not be a general AI, but it says nothing about other models of computation. The notion that (quantum) Turing machines are the most general/powerful model of computation is only a conjecture, although a very plausible one (usually called the (quantum) extended Church Turing thesis).

But even with that in mind, this paper does not seem convincing, including for the reasons you already mentioned. Hopefully it will spark a conversation though.

I am not an academic neurologist, but I have never read anything convincing that suggests quantum effects are critical to the behavior of human cognition. It may yet be true, but I suspect that classical physics/chemistry is probably sufficiently close to what actually happens to convincingly model human behavior.

I completely agree with you. I put quantum in parentheses because quantum behavior is important in complexity/computability, and I wanted to be complete. However, yes, there is absolutely no reason to think a human brain possesses that computational capability (and plenty of reasons to suspect it does not).

How does quantum behavior affect computability?

It does not. Quantum effects are a smokescreen for clinging onto the idea that "only humans can be human". BQP is quite easily contained in PSPACE, and more generally adding quantum operations does not move one beyond Turing machines (realistically - only something like a halting problem oracle does).

I think you missed the point of my comment. BQP (i.e. "stuff that is efficiently solvable by a quantum computer") is almost certainly bigger than P (i.e. "stuff that is efficiently solvable by classical computers"). There was not even remotely anything in my argument that was claiming "only humans can be human" or "only humans can be intelligent" or anything like that (that would have been a silly claim). And nothing remotely related to claiming quantum effects have anything to do with human brains.

It affects complexity. Sure, it is fascinating to learn there is a separation between computable and not computable tasks, but there is an important separation among the computable tasks as well. There are "practically computable" tasks (computable with a reasonable complexity) and "computable tasks that are still infeasible in the real universe" (e.g. tasks with exponential complexity). Amusingly, it seems that there are also tasks on the border between between practical and infeasible, that can be practically solved on a quantum computer, but can not be solved in a reasonable time on a classical computer (even one as big as the whole universe).

P.S. There was some abuse of naming conventions and nomenclature above.

P.P.S. While computability is a well proven concept, claims about complexity (i.e. practical vs infeasible "computable" tasks) are frequently only conjectures (although they have some supporting empirical observations).

P.P.P.S. The proper terms to google for as a starting point would be (really, this barely scratches the surface):

- complexity class P: computable tasks that can be efficiently solved by a classical computer

- complexity class BQP: computable tasks that can be efficiently solved by a quantum computer (and includes tasks that are conjectured to not be efficiently solvable by a classical computer)

- complexity class NP-hard: computable tasks that are conjectured to not be solvable by a classical or a quantum computer efficiently (i.e. a medium sized problem would take a computer bigger than the universe to solve)

I think your premises are fair, but assumption #3 ("Any mathematical rule can be computed by a sufficiently advanced computer") is effectively ruled out by Gödel's incompleteness theorem[1] and/or the Church-Turing thesis[2].

The problem then becomes finding an approach to general AI that avoids hitting incompleteness/undecidability[3] issues. My feeling is that this would be difficult. One way to try to avoid these issues is to avoid notions of self-reference, since self-reference spawns a lot of undecidable stuff (eg, "this statement is false" is neither true nor false). It seems to me, though, that the notions of the self and self-awareness are central to human consciousness, and so unavoidable when developing a complete simulation of human consciousness. The self is probably not computable.

Obviously there could be approaches that avoid these pitfalls, but every year that goes by without much progress towards general AI makes me feel more confident in this intuition. I do think there will be lots of useful progress in specialized AIs, but I see this as analogous to developing algorithms to decide the halting problem for special classes of algorithms. General AI is a whole different beast.

But if general AI is physically impossible, how does the human brain "compute" general intelligence at all? It could be that your assumption #1 ("Physicalism is true. Nothing exists that is not part of the physical world.") is not correct. Maybe reality has "layers" and our world is some kind of simulation in another layer. Or maybe there is only one consciousness like many spiritual people and Boltzmann[4] suggest. Or maybe the human experience could be a process of trying to solve an undecidable problem and failing...

1. https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_...

2. https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis

3. https://en.wikipedia.org/wiki/Undecidable_problem

4. https://en.wikipedia.org/wiki/Boltzmann_brain

In another reply, I modified point three to be the assumption that all of the physical laws of the universe are defined by computable maths. I believe this is the case to the best of our knowledge, but please let me know if I'm wrong.

Unfortunately, restricting to only computable maths means disallowing the natural numbers, basic arithmetic, or any equivalent structure, since Gödel incompleteness would apply. I doubt any system without access to the full set of natural numbers or basic arithmetic could qualify as "general AI".

Pardon my ignorance. Computers appear to be able to perform basic arithmetic. For example, you can open up the console in your browser and find that the sum of two and two is indeed four. So it is not entirely obvious to me how basic arithmetic is non-computable.

If you permit infinitely many integers it becomes problematic. If you are dealing with a finite entity (e.g. the finite part of the universe that can affect us), then there are no problems.

Can you sum correctly two arbitrarily large integers?

I don't think it matters, right? Since arbitrarily large integers are not things that occur in the physical world.

How do you know all those things about the physical world? For example- you say that "all of the physical laws of the universe are defined by computable maths". Do you really know what all the physical laws of the univese are?

We don't know. We just think it likely. We are unaware of counterexamples, or reasons to suspect the existence of counterexamples.

Again I have to ask- who is this "we"?

Apologies if my question sounds too contrarian, but I think you are making some very big assumptions about the computability of the laws of physics that are not really based on anything concrete, like a strong knowledge of the mathematics of modern physics.

We is humanity, as far as I know and as far as brief Googling is able to determine. I am not a physicist, so I have good knowledge of physics up to the high school level, and a dabbler's knowledge of what lies beyond. I am open to correction, so feel free to offer some contradictory evidence if you have any.

What exactly did you google for?

Are we really communicating here? I'm saying that there is a lot that physicists don't know about physics and that therefore it's impossible to make the assumption that you make, that every law of physics is computable. Because nobody knows all of them, and nobody knows what nobody knows, or how much of it there is.

And you're saying that, given high-school physics and "dabbling", we know all of it and it's all computable.

Is that a good summary of our discussion so far?

> Is that a good summary of our discussion so far?

Hrm, I wouldn't say so, and I don't think if that is your impression that it's going to be very productive to continue it.

FWIW: "Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith."

Yes, I'm aware of the guidelines, thank you. They are not a tool to passive-aggressively end conversations by accusing other commenters of bad faith.

But I agree this is not a productive conversation. I don't see that you have a very clear idea of what you are trying to say.

Even if real infinities existed, it would be impossible to tell. What do you measure it against?

Infinite computations are not the only computations that are impossible to perform. For example, if I asked you to enumerate (not calculate) the number of X time units in all the time from the start to the end of the universe, setting X to the closest time unit to the time an operation took on your chosen hardware (past, present or future) you would not be able to complete this computation.

For instance, if the fastest hardware available to you performed about one operation each femtosecond, it would not have the time to enumerate all femtoseconds from the birth of the universe to its death. And that number is a finite quantity.

Gödel incompleteness applies to any system capable of basic arithmetic.

I'm unsure how this matters? The physical universe does not prove itself and does not need to. Godel's theorems just say that certain types of mathematical systems can't prove themselves, which seems quite irrelevant to simulating the universe. Please explain if I'm missing something.

>> But if general AI is physically impossible, how does the human brain "compute" general intelligence at all?

Who says that the brain "computes" general intelligence? We don't know enough about the brain to know what it is, but it's certainly nothing like a computer. Only by analogy is intelligence something that can be computed and the only reason we have this analogy in the first place is because we have computers. But isn't the accuracy of the analogy what we would like to know with some certainty, in the first place?

This is just another big assumption that is taken for granted: that the brain is a computational device. It seems an easy assumption to make, given all we know about computation. And yet, like you say, several generations of AI researchers have failed to reproduce intelligence with computers. Perhaps the reason for this is that the brain is not a computer, intelligence is not a program, and that's why we do not BSOD when confronted with paradoxical statements like "this statement is false".

How many particles make up a human, and how many transistors does it take to simulate the behavior and interactions of all those particles? How much energy does it take to power that many transistors? How does that compare to the Sun’s output? There are a few of the questions we need to answer before confidently predicting that we will ever be able to make a physics-based human simulator.

Yep. This is not a strong proof that humans are simulable, and I don't intend it to be. And I don't meant to suggest that simulating a human particle for particle will ever be an efficient way to model human behavior. I suspect that even medium fidelity neural simulations will someday be sufficient.

Instead, the list is meant to inspire an engagement at the level of: "which of these assumptions do I disbelieve?" For it to be the case that computers cannot mimic human-level intelligence, at least one of these items have to be false. And I consider the last one to be the weakest to attack, because all you're really saying there is that it will never be practical, not that it will never be possible. I think disbelieving any of the other assumptions makes a stronger statement, but I also think they are the harder ones to disbelieve.

I’ve thought along these lines myself, and I do wonder if we’re going to approach general AI by finding ways to “compress” a simulation of a human brain, or just by coming up with new neural net architectures. The second approach is the one everybody’s working on, and to me it seems more practical.