If evolution can cross that barrier just by banging molecules together and seeing which ones work, it seems unlikely there’s some causal disconnect that makes it impossible for us to get there by thinking about it.
Gödel doesn’t say ‘mathematics cannot contemplate itself’. Quite the opposite.
As far as I recall, one of the results of Gödel is that a system capable of Arithmetic cannot prove certain things about itself. Aka there are limits within a system. My claim was about limits existing.
I could understand the hypothesis that AGI is not computable, if we didn’t have an existing example of a machine that can produce AGI.
Since we do have an existing machine that can produce AGI, we would have to suppose it does something:
- outside of physics to achieve its results
- performs some operation that is impossible for us to understand or replicate
Both of those seem… unlikely to me.
Without that assumption, we don't even know that intelligence is computation.
I also don't think it's fair to call it an article of faith, since we have strong evidence to show that _to date_ the universe has followed predictable physical rules. That could, obviously, change at any time, but it seems at least a reasonable prior to assume that the physical rules that we've studied in the past will continue to operate into the future. "The sun will rise tomorrow" is, I _guess_ an article of faith, but I think it's more fair to say it's a reasonable and well founded prediction based on a well studied model of the solar system and the physical laws we've observed the universe follow in the past.
So, my beliefs are:
- The universe is mechanistic, and follows physical rules
- Human beings are also mechanistic, and follow the same physical rules as the universe
- Human beings are intelligent
- Human beings are at minimum turing complete computers (since you could give me a roll of paper, a set of op-codes, and I could perform calcuations)
So, it seems to me a reasonable starting assumption that intelligence is a result of the mechanistic universe. Do we have any evidence for something outside the physical, mechanistic universe impacting human cognition? I'm not aware of any.
But we have lots of evidence of the physical, mechanistic universe impacting human cognition and intelligence.
I'm willing to acknowledge that we don't know that intelligence is Turing Computable, but I'd argue that intelligence is likely to be a mechanical process that's compatible with the physical rules of the universe. Can we be certain of that? No, we can not be 100% certain. But it seems a much more reasonable and evidenced hypothesis than something which asserts there is a metaphysical process that produces intelligence, but which we have no evidence for.
So, no, I don't think it's reasonable to say that assuming a "mechanistic universe" is an article of faith. I think it's a reasonable belief, based on the evidence that we have. What would be an article of faith is asserting that it could _only_ be a mechanistic universe, and refusing to accept any evidence to the contrary. I have not done that.
We're going to get into semantics pretty quickly, but I would argue that at least purely physical intelligence is a procedure that is capable of being run on a machine that exists in our universe.
I would also argue that, such a machine is _at least_ turing complete (since I would think a general intelligence should be capable of emulating a turing machine in the same way that humans are able to do so). I would happily accept that it's possible that being turing complete is a necessary, *but not sufficient* condition for intelligence. That is, I could see a world where intelligence requires some form of hyper-turing machine, that is able to solve some non-turing computable problems.
However, I would argue that even if intelligence does require non-turing computable functions, there exists *a* machine which can perform those procedures (that is, we exist). Thus, in this hypothetical universe where intelligence is non-turing computable, we would then have an existence proof for a hyper-turing machine which can compute non-turing computable results. In this universe, then, I'd argue that anything that can be produced by this "hyper-turing machine", _is_ "physically computable", even if not "turing-computable".
Ultimately, if we accept the premise of a mechanistic universe, I think humans are an existence proof of a machine capable of producing intelligence. Then, I think it follows that it *cannot* be physically impossible to create a machine capable of producing intelligence. Whether we call that machine a "computer", though, I don't have a strong preference.
So, I’ve only used it in a “I understand what it means and what I mean by it” manner in the past, and I haven’t rigorously defined it for myself.
I’ll do the best giving an off-the-cuff definition, but I’m sure it will have some holes and would be improved with some time spent thinking about it, or reading to crib a definition from someone else who has already put in that time.
I think my definition of a mechanistic universe would be:
A universe that transitions from one state to an adjacent state through a consistent set of rules.
From your options:
- mathematical? I’m think yes, but potentially only if “mathetics” is defined broadly.
- Deterministic? I don’t think this is a requirement for a mechanistic universe. My bet would be on a deterministic universe (through an Everettian interpretation lens), but I think something can be mechanistic while being stochastic, so long as the rules determine the probabilities. That is, I would consider a computer with a true source of randomness to still be mechanistic.
> Non-dual with no unmeasurable causes?
Non-dual, yes. That is the primary meaning for me. “With no unmeasurable causes” is a phrasing that makes me slightly uneasy. I could imagine a situation where, if the universe is stochastic you could end up in a situation where you cannot precisely identify the specific cause for a specific event. But, I think as the spirit of this question is intended, yes.
The work he’s doing in that area is super interesting, and sounds promising (of course, I’m getting primarily Carroll’s take on it when I hear about it, so it makes sense that it sounds promising!), but the math itself is above my head.
I am super interested to see if it ends up moving the ball forward on resolving the GR/QM conflicts.
The thing I'd really like is a Turing machine where the nth transition took 1/2^n seconds, so we could run them to infinity in just two seconds.
Problem is, even granted the above, that's not enough of an argument that AGI is only a matter of mechanism.
Take the analogy with pain. We've discovered the mechanism for pain. That it's e.g. some stimulus applied to skin which sets of nerve signals that register in the brain. To then say that the experience of pain is the same as the physical mechanism we've discovered still misses a key step:
what does the experience of pain inhere to, and is that the same subject as that for the mechanism of pain?
Comparing AGI and human intelligence has the same problem. We don't know exactly what is intelligent in either case, let alone whether the two are comparable. So it's not a question of intelligence per se, but of that which is intelligent. Maybe AGI, the way we are thinking and talking about it, is unavoidably tangled with having to grapple with [self-]consciousness.
If I made a careful atom by atom (or quark/lepton/boson) copy of every atom in your brain and body in a different location, would that new copy be intelligent?
If our intelligence is purely a result of physical processes, then why do you suppose it would be impossible for us (or more advanced future beings), to construct a machine which can also follow those processes?
If you think your intelligence is not reducible to purely physical processes, then… what is this meta-physical thing that is required for intelligence? Does it interact with anything other than intelligence, or is it only involved with intelligence? Is there any reason or evidence we should assume this meta-physical intelligence?
I am most sympathetic to the argument that self-consciousness is an emergent phenomenon that is not easily modeled, but is nevertheless derived from physics. In the context of my previous comment, it is this emergent phenomenon that is deemed "intelligent".
> If I made a careful atom by atom (or quark/lepton/boson) copy of every atom in your brain and body in a different location, would that new copy be intelligent?
Yes.
> If our intelligence is purely a result of physical processes, then why do you suppose it would be impossible for us (or more advanced future beings), to construct a machine which can also follow those processes?
I think it is possible, and not too far off to boot. We might get (maybe we already have gotten) the intelligence without the particular emergent phenomenon of self-consciousness that is of the kind humans possess, at least not at first. Some of us will call that AGI, some of us won't. Because we are still working out the terminology and taxonomy. Discussions like these in HN help.
Maybe we'll get the emergent phenomenon soon thereafter. If it is in fact intractable to model (and therefore to train for directly), it may happen unpredictably. Exciting times.
There's a lot of ground to cover between AI as increasingly elaborate magic 8-ball toys and a real human-rivaling AGI. That is, one capable of observing an environment, identifying goals and problems, planning, acting, and reacting. In these much longer (stateful) cognitive chains, there are more opportunities for pathological failure modes and less opportunity for a toy user to charitably excuse the misbehavior.
This is not some kind of dualist metaphysical argument about the possibility of AGI in the abstract. Merely a doubt that we can blindly scale up the complexity of a synthetic mind to meet or surpass our own. Consider that we still can't even begin to understand our own minds in enough detail to reliably predict, repair, or augment them.
I am outside this field and so may have too much of a layman's perspective. But it seems to me that contemporary AGI believers conflate training and evolution. That nature did it in eons doesn't argue that we can do it in practical product development cycles, unless we can simulate these evolutionary processes to follow a similar search in a compressed time scale.
As a crude analogy, I think todays LLM products are a bit like horoscope generators. Clever arrangements of words that attract a charitable or gullible reader. But AGI use cases are more like wanting a life partner who will understand and willingly assist ones efforts.
IMO that is far from clear. In some aspects (multiplying matrixes) it might be more powerful. In others (power consumption) the human brain wins by a wide gap.
The point that brains are more efficient than computers is valid, but that doesn't tell us much about eg whether we'll be able to make a computer do everything a human brain can do within 50 years.
Evolution also crossed the flight barrier by banging molecules together. I don't think banging molecules together without having an understanding of physics and the forces involved would have been a viable means for us to get to flight.
AI/flight analogies are tired, but the OPs argument amounts to the equivalent of, before the Wright Brothers, proclaiming ‘there’s an inherent inability for humans to ever conceive of a way to engineer heavier than air flight’.
It’s a ‘man was never mean to fly, therefore heavier than air flight is impossible’ argument.
So, to date, a mechanistic view of the universe seems like one that is the most supported be evidence at the moment.
Have we ruled out meta-physics or some kind of duality? No, of course we haven’t. As long as there’s unknown physics, it’s a possibility.
But I think if you’re reasoning based on the evidence we’ve been able to collect to date, then you should absolutely favor a mechanistic universe to a dual universe.