Well, when you put it like that, it doesn't not sound like a cult at all.
If I rephrased the quoted passage as:
> transhumanists have deepened their hope for future in which the human species will augment evolution by fusing with machines (as we already do with brain-computer interfaces, but better), and the potential that current AI — which is already helping us decode proteins, perform and write up scientific research — will continue to improve, helping us to expand our medical knowledge even faster and help us develop cures for things currently beyond us. As the AI we already have has allowed us read the complete connectome — neural structure — in small animals, there's even a possibility we may expand this to human brains (though we don't know how long this will take us to do!), and if we manage that, then it would allow us to back up and restore our minds as we already do with digital documents.
Doesn't seem nuts, unless you believe humans have souls that can't be digitised.
That said, I'm skeptical about any specific future: when the only flying machine you've heard of is an airship, it's easy to imagine a future filled with them — what's our airship? What's the thing everyone expects but which won't happen?
The eternal dark that I can't experience doesn't scare me, but I'd still like to have a backup — or at least, I did, then I started seriously considering what else can be done with those backups besides restoring from them…
I probably still would, but I'm much less optimistic about people than I used to be.
And yeah, wait for someone to create 1,000 instances of you to do whatever they want with.
(Today such a thing is totally implausible; perhaps it still will be in 30 years, but I'll only find out by living until then).
I'm sure from your previous comment that you don't see that as a continuation of self, but I do see it as such.
Similar works for any potentially dangerous activity.
A body produces consciousness, consciousness does not 'inhabit' a body.
Let’s ignore souls: humans have bodies which cannot be digitised. Our bodies are real matter, not some dross to be jettisoned.
I wonder if Black Mirror ever did an episode about someone unwillingly personality-cloned into a computer, then killed, with everyone around him acting as though this were normal and he — still conscious — was a non-person?
Some of the stuff that happens to the body clearly impacts our states of consciousness. Blood-alcohol levels, for example. So if I take your statement at face value: OK, so you simulate at least some of the rest of the body, perhaps all of it. So what? Still a simulation.
You can also think of it this from the opposite direction, where someone might try to argue against a purely material origins of consciousness by saying "the brain doesn't work if you take it out of the body" — that this is in fact true, doesn't mean the brain isn't the bit doing the thinking.
You can take Searle's position (why yes, I do have a philosophy qualification, albeit a mediocre one): "No one supposes that computer simulations of a five-alarm fire will burn the neighborhood down or that a computer simulation of a rainstorm will leave us all drenched."[0], but that only works if a mind is a substance of its own kind rather than the emergent result of the processes that gave rise to it. The components of a simulated rainstorm will still interact with each other as those of a real one, if given sufficient fidelity.
Perhaps minds are emergent from some quantum shenanigans, as Penrose says[1]. Then it would take a quantum computer rather than a classical one[2].
> I wonder if Black Mirror ever did an episode about someone unwillingly personality-cloned into a computer, then killed, with everyone around him acting as though this were normal and he — still conscious — was a non-person?
What you describe is a P-zombie, and as https://en.wikipedia.org/wiki/Philosophical_zombie will show, there's no consensus over this even amongst philosophers.
[0] https://en.wikipedia.org/wiki/Synthetic_intelligence#:~:text...
[1] but it would be horrifying if he's right specifically about microtubules, because those are in basically every living cell, and hence if they do mediate consciousness then potatoes would be conscious: https://en.wikipedia.org/wiki/Orchestrated_objective_reducti...
[2] While it's technically possible to simulate quantum processes classically with an exponential slowdown, those kinds of technicality also make the entire universe and everything in it into a Markov chain, I prefer to only point out such things to show the limitations of making such arguments.
If we don’t know what it even is, why is it so inevitable that we’ll be able to recreate it in another medium?
> connectome — neural structure — in small animals, there's even a possibility we may expand this to human brains (though we don't know how long this will take us to do!)
I don't know how to phrase it as any less "inevitable" without saying it's "impossible".
That said, a restore from backup may be easier than understanding/accurately simulating at exactly the correct depth to do whatever the important thing turns out to be and not waste effort on unnecessary things*: we've known for a long time that it's possible to encourage neurons to grow in certain ways with electrical stimulation. At some point in the research process, I expect someone to try to use this to replicate an existing pattern (probably in something simple like a fruit fly) — if that organism seems to remember things that only the original had learned, that would be interesting.
I expect that people will do the same test with the digital version of the connectome and eventually get it to also remember something from the original living brain, and that people will even then continue argue that the the digital version is still missing something. And that this argument will continue even if the simulation is scaled up to a human.
And even if we can not simply store the copy but allow it to develop in the simulation (human brains do change over time), I expect these arguments to continue even if that digital human connectome simulation learns something, then is downloaded into an organic brain in the fashion I have described, and the new living human brain discusses things that only the simulation had experienced.
Right now, on the topic of personhood, we don't even know the right questions.
* in one extreme, a computer can simulate any physical process. Right now, it looks like quantum mechanics is sufficient for all chemistry, and hence all biology, and hence brains. But it may be wildly excessive to simulate all the way down to that level — we don't know, we have much to learn. On the other, neural nets in AI are widely recognised to be toy models, and lots of people assume they're too simple to explain biological cognition — again, we don't know, we have much to learn.