It seems plausible to me that we could create forms of intelligence that only run on a computer and have no bodies. I agree that we might find it difficult to recognize them as intelligent though because we're so conditioned to thinking of intelligence as embodied.
An interesting thought experiment: suppose we create intelligences that are highly connected to one another and the whole internet through fast high bandwidth connections, and have effectively infinite memory. Would such intelligences think they were handicapped compared to us because they leak physical bodies? I'm not so sure!
I do fully agree that any intelligence may not be human-like, though. In fact, I imagine it would seem very cold, calculating, amoral, and manipulative. Our prohibition against that type of behavior depends on a social evolution it won't have experienced.
"physical environment" ? VR lets you operate on and get sensory input based on a fairly significantly degraded version of physical reality.
Seems like you are confusing Consciousness with Intelligence? It's completely plausible that we will create a system with Intelligence that far outstrips ours while being completely un-Conscious.
>I feel like any AGI developed in silico without freedom of movement will be fundamentally incomprehensible to us as embodied humans.
An AGI will be defacto incomprehensible to Humans. Being developed in Silicon will have little bearing on that fact.
Is it? Is there proof that consciousness is not a requirement to close-to-human level intelligence?
Given that we do not even know how consciousness works in our brains, I don't think we have the answer yet. I'm not sure it's plausible to make such an assumption.
Possibly a fair argument, but I think the opposite would be just as valid then. That you can't assume that consciousness IS a requirement for intelligence.
It seems hard to argue that you don't need some intelligence for consciousness, there seems to be a kind of minimum. We can already see that narrow intelligences for Vision, NLP, game playing, etc are at or above human level and no one claims these systems are conscious. So why should we assume that some meta system of these already existing systems would be?
Why do you think embodiment should be necessary?
Supposing it's needed, a universal computer can simulate physics. Now we're just haggling about the price.
It's possible that developing a robot is the most fruitful angle to take on the problem -- giving you the right subproblems in the right order, with economic value along the way. Hans Moravec thought so, writing a few decades back.
Human-level and human-like are not necessarily the same though. I doubt human intelligence is as general as we like to think it is. There's probably a lot about what we consider intelligence that is domain-specific. Training AGI on unfamiliar domains could be super valuable because it would be easier to surpass human effectiveness there.
My pet conjecture, though, is this: the ability to experiment is key to developing AGI. Passively consuming data makes it much harder to establish cause and effect. We create and discard hypotheses about potential causal links based on observed coincidences (ie, small samples) all the time. Doing experiments to confirm/refute these hypotheses is much less computationally expensive than doing passive causal inference, which enables us to consider many more potential causal linkages. That allows us to lower the statistical threshold for considering a potential causal relationship, which makes it more likely for us to find the linkages that do exist. The benefit of developing accurate causal models is that they are much more compact & computationally-effecient than trying to model the entire universe with a single probability distribution.
For example, which language would it "think" in? English? Some other? Something it's own? How would it formulate things? Based on what philosophical framework or similar? What about math? General reasoning? Then cultural norms and communication?
However, AGI doesn't need embodiment in order to be useful. In fact, it might be detrimental. I believe AGI will happen much sooner than embodied AGI.
One is to build something inteligent (an AGI), and the other is something human like. Intuitively we could hit the AGI goal first and aim for human like after that if we feel like it.
In the past, human like inteligent seemed more approachable for I think mostly emotional reasons, but for our current trajectory if we get anything inteligent we’d still have reach a huge milestone IMO.