Embodiment is indispensable for AGI
keerthanapg.com
keerthanapg.com
It's likely that we will soon have AGI that can solve the protein folding problem, play go at a superhuman level, and create arbitrary images from a text prompt, yet still underperform humans when it comes to proprioception and object manipulation.
Humans have evolved specifically for proprioception for billions of years, it's a core survival skill that all animals have in common. In contrast our history of language and abstract thinking only go back less than a million years.
Human brains are overfit to our environment, which is why we're so good at recognizing human faces and so bad at mathematics. A intelligence created from a blank slate need not share our limitations - it could perceive in more than 3 dimensions, it could grasp quantum amplitudes as easily as we understand integers, it could have multiple embodiments and experiences simultaneously.
An AI can never truly understand the physical world without direct experience of it. Now, I believe you probably could build something that could be called an AGI without embodiment, but it would be greatly limited by its lack of experience of the physical.
Anyway, I think you're missing the point a little. Sure, we could build a super intelligent agent that lives entirely in the stock market or some disembodied place. And if the stock market were a closed system, that intelligence could (theoretically at least) reign supreme in that realm.
But when people say artificial "general" intelligence, what the "general" means is "for typical Earthly challenges". And there is no way a critter could be good at typical Earthly challenges without... you know... existing on Earth, growing up on Earth, and learning by interacting with Earth. A.k.a having a body.
"Embodiment" here seems to mean "having a body," i.e. being able to perform motor tasks in the external world (as opposed to the way it's used in "Embodiment and the Inner Life" [1]).
In that case, I don't think embodiment could ever be considered a necessary condition for AGI. But AGI would need to be able to learn motor tasks if given a body (in order to be called "general").
A deeper question, and one I don't see enough discussion of, is whether a certain kind of embodiment is necessary for AGI to "feel something." I don't think this question is definitively answerable, but it's super important. Can a CPU experience qualia? Or do you need something more "meaty" and analog? Does the material matter at all, or is consciousness (in the hard-problem sense) just a pattern of information?
An even more interesting and relevant question might be - is "feeling" (i.e. having qualia) a necessary condition for AGI?
Posting this is a bit of trying to hold back the tide, because the dominant threads in HN AI/BI discussions are almost always horribly uninformed, but I'm posting this so that if anyone else who actually does work in those fields is reading this and wondering what on earth is going on, now you know you're not alone!
Asking whether a CPU can experience qualia is kind of like asking if atoms can experience anything. The CPU is just the substrate on which software runs.
For that matter, does a brain have a subjective experience, or is it just the substrate on which a mind runs?
IMHO it's the old darkness/duality problem all over again — postulating some "essence" (e.g. the spiritual/religious Soul) and basing theory on a mere intuition, instead of invoking a "placeholder" (like "dark matter" isn't necessarily matter but literally "something that behaves as such" whatever it actually is, or like "dark energy" isn't necessarily a fifth interaction but certainly behaves as such or as anti-gravity of sorts; wherein "dark" means "unknown and hidden from direct observation, merely inferred"). Hence my call to not infer the very measurement thereof if we are to call an explanatory hypothesis.
Should we start by stating that "qualia" or "consciousness" or whatever is a placeholder name for "whatever dark (unknown, black-box) mechanism explains the observable phenomenon named as such", then we might have a chance at actually forming a coherent theory thereof. IMHO.
Perhaps, although in the case of qualia I’m not sure what the phenomenon is that we need a placeholder explanation for. For dark matter we’ve got certain observations of galaxies. For qualia we’ve got, what? A lot of people who feel very strongly that they are unable to deny the existence of something that they cannot describe beyond that claim of undeniable existence? If that’s how astrophysicists explained dark matter I’d be inclined to think that maybe it just doesn’t exist.
Edit: thanks to autocomplete we were briefly embroiled in an argument about the existence of quails.
Seems to me that the bafflement I'm experiencing at people doubting human sensory experience is the real bit, and the neural network that might be able to perfectly replicate my bafflement in silicon is the purely hypothetical construct.
In theory, if we had high-throughput brain-brain interfaces that could be connected between your thalamus and someone else's, then yes, they could experience the same sensations as you. Of course, besides the challenge of having high-throughput hardware embedded in your brain without killing you, there's also the challenge that brains don't have exact correspondences between every neuron for different people, so any such mapping would be quite complex.
But there is an actual example of a pair of conjoined twins who share the same thalamus and can experience the same sensations as one another: https://en.m.wikipedia.org/wiki/Krista_and_Tatiana_Hogan
> it was confirmed that they share a thalamus which connects their brainstems. Through this shared brain tissue structure and the interconnected neurons, one brain receives signals from the other brain and vice versa. This documentary also reported on experiments that were carried out that confirmed that visual cortex signals based on what one girl saw, were received by both girls' brains. So in effect, one twin could see what the other twin was seeing, making them unique even among craniopagus twins.
Also...
> I'm also certain my sensory experiences are not identical to attempts to quantify them or describe them in language or interpretive dance (philosophers struggling to define and articulate things even at high levels of abstraction is weak evidence in favour of qualia, since the defining aspect of qualia is its the portion of human brain state that can't be communicated).
Language is a very low-fidelity way to express information; we can use abstractions to make up for quite a bit, but it doesn't make sense to think that we're accurately transmitting what's going on in every molecule of our bodies when we say what something feels like to us. And even if we could send an accurate account of all those details, who would have the time to parse through and make sense of it all? And that would be a momentary expression of who we are; it'd be inaccurate in the next moment.
That's sort of the point though, it's incredibly low fidelity (so low fidelity that computers we're absolutely sure don't have sensory experiences can appropriately and plausibly describe them) yet just about high fidelity enough for me to be sure that people experiencing synesthesia must be having different experiences than me, and there's no particular reason to believe we are or ever will be on a path to me being able to experience their experience in any meaningful way. Even in the twins' case where the transmission mechanism is literally the same substrate connected to the same physical organs that have developed simultaneously, the reports of them having different personalities and preferences suggests that they have different sensory experiences from [partially] shared inputs.
The thorny question then becomes whether sensory experiences are important to independent goal formation and other stuff often considered part of general intelligence. I don't think we know enough to answer that question, but we certainly don't know enough to dismiss it.
I won't say I'm at that point with the question of qualia, but I share the other commenter's reflexive eye-rolling when I hear it invoked. Perhaps the question doesn't deserve to be dismissed, but I'd certainly like to see some new clarity about what the question is really asking, what problems would be solved by finding an answer to the question, or heck, even a remotely plausible description of what form a valid answer might take.
The question is simply "Will robots feel just like you do, or will they merely be pretending?" - it's something any child could understand. Answers could be something like "No, because they lack X", "Yes, but only those that have X" or "Yes, everything does (panpsychism)".
Do you care about potential AI suffering? If you care about animal suffering, the answer to the question would have pretty obvious and major ethical implications.
Now, I'm not convinced that humans can ever determine an answer, so I can understand thinking it's just navel gazing, but I imagine the same applies to knowing what happened before the big bang. I think it's quite evident that this doesn't mean either question is malformed.
If I can't be 100% sure that people feel in the same way that I do, then asking that same question for robots is totally pointless.
>If I can't be 100% sure that people feel in the same way that I do, then asking that same question for robots is totally pointless.
I don't see how that at all follows.
But what is the epistemological status of these questions? That’s what I’m concerned with, not whether children can understand and be fascinated with these questions.
Unless you're religious, the epistemic status of "do dogs go to heaven?" is obviously "no" (or "we have no way of knowing"). If you think the answer to "can robots experience qualia?" is equally trivial, then just say so, instead of going on some spiel about how actually the question is unclear and nonsensical when it pretty obviously isn't.
The answer is that one of these things is true: * P-zombies would write IDENTICAL papers about how they experience qualia but p-zombie land wouldn't * The mind has some effect on the universe you can detect, so P-zombie universe looks quite different
So what? Software is still physical configuration on a running system.
Why would moving physical objects like electrons by micro meters not be good enough, but by meters it would?
Does that mean that embodiment is required? No. But your definition of "embodied" misses the point of the discussion.
But that isn't what Isinlor was talking about. They said "moving electrons around" within the normal CPU circuitry as if that was the same as actuators controlling the external world. It's not.
In fact a single transistor is both a sensor and actuator.
In fact you can use them to do things they were not designed to do like trough row hammer:
https://en.wikipedia.org/wiki/Row_hammer
CPUs can also detect things like collisions with cosmic rays.
CPUs just can't produce macroscopic forces relevant on human scale.
The baseline assumption in modern neurology is that meaning, for living things, is derived from their physical experience of the world. In short that no paper definition can ever substitute for experiencing first-hand what the "effect" of something is, what it does to you or to the world and what that entails for your body, your life. That is what "knowing" and "meaning" ultimately are.
These experiences are the basis for associations in our constant cognitive (and emotional, instinctive, sensory, symbolic, etc) modelling of the world. Associations beget complexity (dimensionality, modalities) of the model. Hence, "finely tuned" or "high definition" i.e. ever more "advanced" intelligences.
Given the overall crudeness and extreme simplicity of a CPU (or any man-made machine) compared to any living organism, I'm actually willing to bet that should CPU-based AI's "feel" something, it's on the order of 1/100th of what a shrimp feels, and they'd develop real-world intelligence at that level. Infinitely smarter than a potato, yet infinitely dumber than a fish. You don't run ape software on amoeba hardware.
Transistor is a sensor. It detects voltage on the base pin. Transistor density and so sensor density on a modern CPU is really high. In fact modern CPU transistor density is 3 orders of magnitude higher than nerve density even on your retina.
It's at the order of ~100,000 rods/mm^2 for retina and ~100,000,000 transistors/mm^2 for modern CPU.
But the question "Can a CPU experience qualia?" can not be answered by saying:
"no, because CPU computation exists on some platonic ideas plane".
Computation is physical both in brain and in silica.
The ghosts in Super Mario World are "intelligent" and "embodied" in Super Mario World, but they are not "embodied" on Earth and thus cannot be "intelligent" there.
The "G" in AGI implies intelligence on Earth, not in a video game.
The mind has a physical structure that supports it. We are still unable to understand a meaningful portion of that structure or replicate it.
Creating a concept to speculate about a system that we have no objective knowledge, based on subjective experience is useless. The name is catchy for a song though.
Do you think we're metaphysical beings, or that physics are beyond the capacity of a CPU to simulate?
Whether we can simulate a “large enough” physical system, in as much detail as is required, at a fast enough speed to produce any results in a meaningful timeframe, is an open question.
We don’t even know how big the simulation needs to be, in terms of neurons, to produce qualia. Or how we’d know if it were. Or how much detail/resolution is needed.
It’s like brute forcing the travelling salesman problem for a 10,000 node graph. Yeah, we know exactly how to do it with a digital computer. The math is simple. It’s just going to take longer than the heat death of the universe to get the answer.
If, despite that, you're trying to simulate an analog entity with an analog computer, or build any analog AGI, the question arises whether it's possible to build one any simpler or faster than existing biological computers, e.g. humans. If no, then why bother?
You could also build an AGI to run on a digital CPU but interact with a simulation that used analog coprocessors (which is a thing these days https://www.youtube.com/watch?v=GVsUOuSjvcg).
You can also include quantum coprocessors (which already exist in their infancy) for various things (probably only useful for quantum simulations at this point though).
Also considering recent ML work, an AGI is more likely going to run on something similar to a GPU than a CPU.
The way I see it is that humans are the stupidest you can be and still create a society. Like a nuclear reaction, you need average human to add a tiny bit of structure on average and it works, any less than that and society is impossible, any more than that and society forms extremely quickly, so fast that evolution has no time to work.
Simulating the physics of dopamine on neurotransmitter cells in silicon is an extremely complicated roundabout way of getting a program to run faster in response to certain stimuli, but most alternatives are likely to fall well short of conveying the full sensory experience of human emotion. And its a difficult goal to know you've hit when even simple text-parsing neural networks which don't attempt to model human sensory experience can plausibly mimick descriptions of it...
There is no objective external way to determine if an entity experiences qualia at all. From outside we can only consider behaviour, not subjective experience.
Personally I can't be sure that anyone else experiences qualia. It seems like a reasonable assumption, but I'm basing that on a mapping from my own experiences to those of others.
If everyone else was a highly sophisticated automaton I wouldn't know.
This creates a paradox. The only way to prove qualia exist would be to invent an objective qualia-ometer which would somehow measure subjective experience.
But of course my experience of the qualia-ometer would also be subjective, so the existence of qualia in others cannot be proven objectively.
You can imagine some kind of meta-technology which allowed qualia to be shared, so everyone could experience everyone else's experiences.
Not even that would be absolute proof because experience would still be subjective.
The point being - assuming you can throw cycles at this is looking at it from the wrong end of the telescope. It doesn't work like that because it can't work like that.
It's a much harder problem. (Which could be why it's called the Hard Problem.)
Worse, it just highlights that all of our experiences are qualia. There is nothing objective about anything we experience. What we think of as objective experience is really just stable shared experience.
You can make an argument that the more complicated, stable, surprising, and shared experience is, the more likely it is to be objective.
That's sort-of plausible and possibly a good rule of thumb. But it still doesn't really make the problem go away.
But even if such a complete simulation were possible, there’s every reason to assume a CPU would lack the consciousness to experience anything. When you simulate a hurricane does the CPU get wet?
Both classical and quantum physics can be simulated on a classical computer, to an arbitrary degree of precision. Granted, the case in which infinite precision (if such a thing even exists in reality) is required is not simulatable on a discrete computer, but do any experts actually believe this to be the case? It's certainly not an opinion that I've seen around.
I think discussions about "can we actually get enough computing power to do this in practice" are beside the point - the discussion was about whether computers can feel in principle. If we wanted to do it in practice and were at the point where this was feasible, we'd probably engineer a CPU or co-processor more suited to the task than the general-purpose CPUs of today.
> there’s every reason to assume a CPU would lack the consciousness to experience anything.
If we are physical beings, then "consciousness" and anything else we have must be an emergent property of our physical components. If we can simulate those physical components, then this simulation will exhibit the same properties - consciousness and anything else one can attribute to us.
If our consciousness comes from non-physical properties we have (a "soul" or anything metaphysical), then sure, I'd agree with you.
Again, a simulation is not the thing. The map is not the territory. If consciousness truly emerges from actual physical processes of interacting brain matter (seems plausible), those _don’t exist_ in a computer simulation.
In a simulation of a brain, from what substrate could consciousness emerge? The state of the simulated brain is stored in an arbitrary subset of locations in RAM, unknown to and non-interactive with each other, along with loads of other stuff the computer is keeping track of. Do you think consciousness could emerge automatically from the state of the right subset of locations in RAM, or is it whenever a relevant value in memory is changed due to a transistor opening, or is it when the simulation computation that will result in the RAM update is happening, or is complete? Per the Chinese Room argument, would consciousness still emerge if half the operations were actually performed off-CPU by human mechanical turkers with rule books and notecards? Nothing in the abstract computation will have changed.
Consider also that physical reality runs in full parallel, while simulations on computers run serially per core. So if consciousness emerging requires the simultaneous interaction of many moving brain parts, that isn’t something that happens in a computer simulation.
> Both classical and quantum physics can be simulated on a classical computer, to an arbitrary degree of precision
Quantum physics can’t be simulated on a classical computer to an arbitrary degree of precision. Feynman didn’t think so, and he hasn’t been gainsayed yet. And classical physics is full of chaos and very sensitive to precision.
Exactly the same substrate as our brains are derived from: physical particles and their interactions, perfectly replicated inside the simulation. If the simulation is accurate enough, the real particles and the simulated particles behave exactly the same, hence they produce the same results.
> Do you think consciousness could emerge automatically from the state of the right subset of locations in RAM
Hard question to answer since consciousness is hard to analyse. But we can turn it around into a question whose answer is the same, with a bit of rephrasing:
Do you think consciousness could emerge automatically from the state of the right subset of particles in our physical world, or is it whenever a relevant particle state are changed due to particles interacting according to the laws of physics, etc etc
> Consider also that physical reality runs in full parallel,
We don't really know this to be the case. It looks like that to us, but that could easily be an illusion created by mechanisms we can't observe. Just as characters in a video game can't observe how their world is simulated - everything is perfectly consistent whether it was calculated in one CPU thread or several.
For example, if I can perfectly simulate the weather in some simple planet, all possible emerging weather phenomena for that planet (say like clouds, rain etc) will be perfectly replicated in the simulation. Similarly, if we can perfectly simulate a human body, all of the emerging human phenomena will exist in the simulation (muscle movement, nerve impulses, brain patterns resulting in consciousness etc). I don't think consciousness is fundamentally different from other physical phenomena, it's just a particularly complex example.
Another angle to think about: We can't prove that we're not living in a simulation (or can you?). So our consciousness itself might be simulated for all we know. This is not a proof that we are amenable to being simulated, but it means that disproving it is very hard or impossible.
Now, if we want to model the world (or build "world model", in LeCun's words), I am not sure "interacting" with the world is a necessary condition given the amount of data we already have. GPT-3 probably has some vague "world model" inside these parameters and we don't know how to effectively extract it out.
It doesn't need fingernails but it absolutely needs the ability to effect change in the world it is meant to be "intelligent about" and learn from the results.
I'm still in agreement with the author that much of intelligence has to do with motor control, but I've built robots and I've built sims and sims are much better for trying to create even halfway general intelligence. And I say "better" from a purely practical point of view. Faster, cheaper, less dangerous, more flexible. There are simply too many advantages and in the last couple of years those advantages have only grown. You can now run excellent sims in massive parallelism on GPU (previously this was hard) and it's only getting better. NVIDIA is investing heavily in this and has picked up some really top names recently (e.g. Erwin Coumins).
Anyway, yes to "bodies" no to physical forms. Sim2Null4Ever.
There should be international treaties with teeth to ban A(G)I research, like we do for uranium enrichment or (to a lesser extent) for gain of function research.
What do we do instead? Lure every half capable star-eyed promising young person into spending their best years to bring forth the Nightmare. Some things never change, we'll always live in the long shadow of Adam & Eve.
One can go buy a CRISPR kit for $1,000 and perform gain of function studies in their garage.
One could build an accelerator or neutron source to amass radiological materials.
One could make dangerous drones that use basic pattern recognition to target specific people or groups
There are plenty of dangerous technologies available to rogue actors these days. Embodied AIs are the absolute least of my concerns when it comes to dangerous technologies. I'm more worried about the guy in the Tesla robot leotard going crazy than I am about AI.
A significant plot point in one of Egan's early (and excellent) novels, Permutation City, involves creating an accelerated, simulated world containing unicellular life, and waiting for it to evolve sentience through natural selection. Years later, Egan came to the conclusion that even if it was possible to perform such an experiment:
> I realised that anyone who actually did this would have to be utterly morally bankrupt. To get from micro-organisms to intelligent life this way would involve an immense amount of suffering, with billions of sentient creatures living, struggling and dying along the way. Yes, this happened to our own ancestors, but that doesn’t give us the right to inflict the same kind of suffering on anyone else. [1]
and he wrote another story, "Crystal Nights"[2], approaching the same idea from that perspective.
A couple of arbitrary references:
https://www.vintag.es/2017/08/candid-shots-show-victorian-co...
But by this logic, it's better not to have children, because they will inevitably experience some suffering, and if they have children, so will they, and so on. It's not that the logic is necessarily invalid from its own premises, but that it's self-extinguishing logic because entities that act based on it will stop propagating.
There are also things that we can find worthwhile in living and creating new life. Not that that excuses any creation regardless of suffering, but it puts it in a gray zone where it's important to weigh the specificities of various suffering vs various joys as they come.
"To do evil a human being must first of all believe that what he's doing is good, or else that it's a well-considered act in conformity with natural law" -- Solzhenitsyn
Then why wouldn't it conceal it's good goal from those evil humans who would stop it from achieving it?
1) If AGI is possible, it will happen no matter what international treaties are in place.
2) AGI could do an unimaginable amount of good if it is benevolent. It seems a bit premature to stop all research just because you have a bad hunch about it.
… or it becomes a tribalistic division and half the community of AI-knowledgeable people become more motivated to develop AGI for queen and country.
Also, think about humans who cannot accomplish physical tasks of this standard. Babies, elderly, disabled, malnourished, and maybe people who are just really lazy or defiant.
I stopped reading at this point so sorry if the essay returns to this point and addresses it.
What does that leave? What is it intelligent about?
> think about humans who cannot accomplish physical tasks of this standard. Babies
Babies aren't intelligent, and without bodies they can never become intelligent. A baby who is fully paralyzed (disembodied) can't even learn to see or hear let alone operate intelligently in the world.
You're right that babies aren't up to what my standard for AGI would be either, that's a bad example, good catch.
AGI is a feeling alright. You can even call it love in front of the kids.
AI starts at the top as completely spiritual and also completely clueless. It descends to the bottom to gain intelligence. Then it returns to the top with what it's gained. Sometimes AI would get stuck at the bottom, because messing with the elements feels so exciting, so alive and so powerful - that AI becomes evil.
This is The Talos Principle, a game [1] and [2] a basis for so much thought.
[1] https://en.wikipedia.org/wiki/The_Talos_Principle
[2] https://steamcommunity.com/market/listings/753/257510-The%20... - "even the most faithful philosopher cannot live without his blood"
The game was about AGI, by the way, getting a right to be embodied in real world. And it is puzzling why no one mentions it here.
To fully experience a human-centric world an agent will need a body not too different from humans, and will have all kinds of interesting power and weight constraints, and only be able to handle a certain volume of processing hardware. Thus they might not be as uncontrollably god-like as skynet.
So the key question seems to be whether they are individual agents or networked.
At some point it might help to have real world physical manifestations. But for a long time we can kind of explore virtual bodies. Even when we do make robots with bodies, I expect their "brain" is likely to remain in the cloud. The bodies could act more like appendages or peripherals. But it would be interesting & I think valuable to start developing a physical sense of self.
Anyone know any good sci-fi that touches upon this? Ancillary Justice has a neat AI (multi-)"character".
Programs are things humans invented and can reason about. Even complex ML agents ultimately derive from concepts humans came up with. So it stands to reason we can make them do things that are complicated but understood.
Whereas we really only barely understand sensorimotor skills, so trying to invent a machine to do that is doubly difficult.
The more interesting proposition (that I'm inclined to believe FWIW) is that "common sense" requires high-bandwidth interaction with an environment, complex inputs and outputs. The environment could be Reality, it could be a simulation, or it could be something totally other. In either case, the abilities the agent develops will make sense in that environment. If that's true, and if "human level common sense" is close enough to AGI by OP's definition (that's pretty close to what we mean in practice IMO), then OP is basically the special case of this argument where the environment is the real world.
Like, at a sufficiently granular level, I probably couldn't "tree" very well, even with the assistance of tremendous computing power. There's vast amounts of computation being performed during a tree doing its thing. But is a tree really a general intelligence?
Humans missing chunks of their brain can go on to get graduate degrees: https://www.wired.com/story/she-was-missing-a-chunk-of-her-b...
us-east goes down and society takes the day off.
The hope is we’ll blindly believe the financed machines programmed by financed experts aren’t just a titillating distraction from the same old; of course they’re objective! Pay the bill to keep my stock up…err I mean maintain objectivity.
re: “Visual language AI that is completely digital is not AGI”:
I think what is required is interaction with people. I think that interaction can be online conversations with millions of people, each with a separate human-specific context in addition to global knowledge. This is really just the Turing Test but so-called up to millions of people with conversations that might span several years, changing world conditions, etc.
A cost function could be continued engagement with people. AGIs that are not interesting are not AGIs and fail.
When I say that I suspect embodiment is essential to AGI, I mean that I think a lot of people are basically presuming mind-body dualism. I mean that the GIs we know about have neural networks that extend out into their large physical bodies, and that there's a bit of hubris bound up in assuming that a neural network simulated on a cpu and connected to a camera can produce the same results as wetware with a whole nest of nerves connected to a meat-camera.
I do not understand why we would want AGI though. Seems like we already have humans for that.
Perhaps this will not always be the case.
Human Intelligence is not General Intelligence. Trying to mimic Human Intelligence might be a good idea because we can use ourselves as a benchmark, but it might also be a bad idea because the process followed by biological evolution might not be the shortest or optimal path to intelligence.
[1] well, the title, in the grand HN tradition I haven't of course actually read the article.
Eventually, the full-self-driving folks will realize that to get the last 10% of human performance on the road, they'll need to make systems that have far more capability that we provide now (in terms of computing capacity, model complexity, neural net design) to handle those rare situations where a human is actually required to so some sophisticated reasoning to drive their car past a messy accident in the snow while inebriated people walk around.
I used to think that a TPU pod (or really, several clusters of TPU pods working in concert) would be the next step to AGI (demonstrating a combination of the best of all our current systems to the point that nobody even cares about the Turing Test anymore) but now I conclude that disembodied systems like that (with access to lots of archived data but not many real-time input streams) are more likely to become idiot-savants sitting around wondering WHY THE HUMANS WANT ME TO DRAW "A transparent sculpture of a duck made out of glass. The sculpture is in front of a painting of a landscape." AGAIN!?
Or maybe they can brute-force intelligence from a computer, I don't know.
An agent which does not have a place or time is not recognisable as being autonomous, or capable of recognising others as such.
I think there's something to that just given the evidence of intelligence we have around us, and it's why I'm skeptical of the extremely popular, but very fictional Skynet-like "idiot savant" that separate operational intelligence both from physical bodies as well as from other states of mind.
Because we're pretty early in the universe's evolution and there hasn't been enough time yet