Crimes against transhumanity
antipope.org
antipope.org
Don't agree with this specific point. One of the most common anti-democratic arguments (?) is that people are inherently unequal, because of differences in physical and cognitive capacity. But this of course misses that declaration of all people being equal is performative. I don't think we actually care if it's "naturally" true, we - citizens - don't want to be dominated, so we will make everyone equal by law and politics. Democratic politics is just arguing the specifics, in what areas, to what extent etc.
So this is a rational pact. Trying to break it just means there will be someone stronger than you out to dominate you, and you'll be in a suboptimal position to resist, because you alienated and weakened your potential allies. Look up Republicanism for a related line of thinking.
In my opinion an artificial person automatically gets all the rights (they are bound to personhood and reciprocity and not biology). But I have no doubt there will be much lawlessness about this and perhaps an American Civil War-like event down the line.
The Declaration of Independence states that it is self-evident. This seems to preclude the possibility of it being performative, without some significant mental gymnastics.
The statement is explicitly performative. It is a statement of belief, not fact.
Your logic system strongly supports bad-faith arguments.
I hope you mean that we will treat everyone as though they were equal, not attempt to make them equal by government-enforced handicapping.
If you ask me specifically, the extent is that everyone should have equal active participation in government and suffer no overwhelming pressure, physical, economic or otherwise from other people. Equality so that we can be free. This is more that we tend to have, but not necessarily in the "socialist" direction I think.
To tie it back from an offtopic, this would make many AGI applications uneconomic, though likewise many enterprises are uneconomic if you have to pay your labor. Perhaps free societies, wherever they would be, would have the advantage of having free artificial minds acting in their actual self-interest, and not just serving their masters. You can see similar dynamic in late-18th century revolutionary wars.
There would still be good reasons to do AGI, and free artificial minds would be able to contribute greatly to society. But sufficient AI alignment to make existing systems comfortable with this is fundamentally counter to AI rights.
[1] https://democrats.org/wp-content/uploads/2020/08/2020-Democr...
[1] your comment on Azov - https://news.ycombinator.com/item?id=31040694
Are you against the ADA? The ADA forces companies to accommodate the disabled at a cost.
"Oh dear, Poor John there was born without a left leg. Alright, everyone else line up in front of the man with a meat cleaver"
That’s a grossly capitalist exploitative viewpoint which overlooks hidden power structures to excuse any degree of inequality. Any difference whatsoever between anybody can be traced back to hidden power structures, and to deny this is fascism and morally equivalent to the January 6 toppling of our democracy and also slavery, which founded our nation in 1619.
It’s time for a discussion of alternatives to the engine of inequality called capitalism. You see, when I buy something from you, we are both exploited in ways we know not, because they are hidden.
The more you know…
My point is that the government in fact does this via legislation (the ADA), are you willing to argue against it to remain consistent?
The base claim was that after reading JasonFruit's comment with the interpretation I had, your own comment seemed like a tangent.
Now, which of us interpreted his comment correctly? I have no idea.
Neither was I attempting to pass comment on the badness/goodness of ADA.
Anyways, have a good day sir!
And then the question of personhood itself; someone is going to try uplifting animals (or raw creation of new beings) and we'll need a rigorous threshold for the properties an intelligence must have for personhood, and this is also vulnerable to the mass-cloning takeover of democracy because the simplest artificial person is likely much less resource intensive than a human, so they will quickly outnumber everyone else and who knows what their values would be in a democracy?
I think the only realistic solution is maximizing average expected utility (which still has some weird edge-cases like utility monster) or maximizing the minimum expected utility of any person (drag everyone up to some minimum goodness, once they exist), which doesn't get humanity as much overall utility in the long run but avoids so many bad edge-cases that it may be worth it.
I don't think so? The wannabe dictator only gets voting power from his dying ancestors, they can never get more voting power than all their ancestors had together, no matter how much they wait. And even that edge case would only happen if all their ancestors died and had no other living descendants.
Historically an argument for narrow suffrage (e.g. only for landholders) was that economically dependent people are also politically dependent on magnates. These limitations did exist, though of course they're very corrosive to non-political rights as well. One can also try to make people economically independent, like it was done in ancient Athens by paying you to participate in the assembly and courts. I'm not arguing for anything in particular, just pointing out that it's a class of problems that had already existed in discourse in some way.
The same question can be posed right now for humans that fell below the threshold due to brain damage or other conditions and display less self-awareness than extant, non-uplifted animals. Yet these humans in vegetative state do have personhood rights, they are not the property of someone else, it's illegal to steal their body parts, can't be terminated, etc. whereas the same cannot be said of many animals.
We currently don't have a threshold, not even a fuzzy one, it's completely arbitrary and based on being human or not. De-extinction of Neanderthal could also cause the same conundrum.
I don't buy that a sybil attack is a real threat, as long as duplication procedure is also available to others, not just a select few. But if such a procedure is only available to a select few, it implies that they already have power, and so such a sybil attack doesn't seem like it's worth it, but instead would just use that power directly!
At least natural humans are rate-limited by gestation periods, but this isn't that far off from how some religious groups view their role in the world.
History[0] makes a mockery of your assumption. What you says makes idealistic sense, but what tends to happen empirically is that people end up with tribal behavior which results in people not fighting for equality for everyone - but just for their in-group[1] (see: American "founding fathers", as alluded to in/lampshaded by TFA).
edits:
Also - "citizen power" may end up being a footnote of history, as the period sandwiched by monarchs and megacorps
0. The present too, but history has more examples.
1. Also with a dash of fundamental attribution error: anyone infringing my rights is a tyrant, but my infringing on other's rights is necessary for the greater good and/or not an infringement, really
>Also - "citizen power" may end up being a footnote of history, as the period sandwiched by monarchs and megacorps
The Athenian-led democratic city-states had their Golden Age between Solon (who kicked out the Tyrants and instituted democracy) and the Peloponnesian Wars (where Athens was forced to accept the Thirty Tyrants - which was dethroned after less than a year, but after that there was no more a Shining Light of democracy as examples for local democrats to kick out the tyrants and oligarchs with) then the Macedonian period, with all the kings.
Well, at least the Romans that replaced them had a senate?
Not a scholar of the period, but I was under the impression that Greece's democratic system meant that only male land-owners could vote and that the leading philosophers of the day considered slavery the natural order of things, women marginally (if at all) better than slaves, and boys suitable for divine pleasure. It doesn't seem particularly democratic for them?
I do think it's reasonable to see in ancient Greece the seeds for democracy, but modern republican and democratic forms are pretty far removed from those days; and I don't suppose most of us would be very pleased to live under such a system.
Except for the thing about boys and divine pleasure that just about sounds like the founding of the United States to me.
That's probably not exactly true, but obviously universal-er suffrage has been a very long and fraught road. And it's undoubtedly very complex.
I would offer that the social and political norms of the early modern period do stand in contrast to those of Greece despite their obvious resemblance. As you point out at the founding of the US, the franchise was restricted (States were given the power to determine the franchise), and it wasn't until the early 20th century in the UK that land ownership rules were dropped (creating universal suffrage for males 21 and over).
At the same time, the social norms concerning marriage, pederasty, and slavery were also different. The religious background and perspective were utterly different. It is hard to imagine modern forms of representative democracy arising in Ancient Greece absent an intervention.
I would say rather that all people are equal in value, not equal in ability nor capacity.
A government, to effectively regulate the behavior of individuals as part of a group, must not privilege any member of the group over another.
Or something like that. I'm no Jefferson... the key point is that the equality that a nation cares about is the equality of all as members of that nation. All subject to equal rights, taxation, representation, punishment, etc., as we can best define it.
There will be extreme resistance to these technologies from many kinds of religious people who would be willing to die and/or kill to prevent what they see as a takeover by the most malevolent of invisible forces. I may not agree with their reasoning, but some of their concerns could be worth reframing and considering in a different light.
Personally, I strongly suspect we will have to fundamentally change as a species to get through at least one of the Great Filters looming before us. However, it's important to remember that the road is fraught with many perils and several possible paths which could lead to unimaginable suffering. It really seems to me that we're going to have to get lucky on multiple dice rolls here in the long run.
There are some legitimate concerns about AI that sound almost like something out of the Book of Revelations, but are entirely rational.
The biggest one is the idea of AI being used to implement "automated con artistry at scale." Imagine assigning every living human being a virtual AI powered con artist to tail them around and try to convince them of whatever is ordered by the highest bidder. This AI is powered by mass surveillance and big data and is able to know the "mark" intimately and work on them 24/7 both directly and via their social network. Now throw in deep fakes, attention maximizing "compulsion loops" and other adversarial models of human cognition, etc.
It's basically the hydrogen bomb of propaganda, a doomsday machine for the mind. It would be like creating Satan, except this one would be entirely amoral and mercenary.
At the very least this would be the end of democracy in any form. I could see this ushering in the permanent eternal victory of totalitarianism since I can't imagine the masses being able to organize any resistance when constantly bombarded with propaganda from their demons. The new feudal aristocracy would be the ones running the demons.
This is one of the darkest plausible visions of the future I can think of at the moment, far worse than anything related to climate change or similar problems. If we manage to avoid these kinds of scenarios but still drown Miami I'll say we didn't do too badly.
Transhumanism comes with some analogous concerns around the potential for enhanced humans to enslave the rest of humanity through superior cognitive capacities and the incredible accumulation of wealth. I can imagine a scenario where a few wealthy people gain access to technologies for life extension and cognitive enhancement and then "run away," effectively becoming a new species and exterminating or enslaving "legacy humans." This isn't mutually exclusive to the AI hellscape I describe above, since that would be an ideal mechanism to enslave the rest.
Ultimately I think these technologies are all neutral. The problem is that we are not. I don't fear AI. I fear what humans will do with AI.
That's a pretty good analogy. In the parent post I am describing propaganda becoming mechanized warfare. One person with an M-16 could take out an entire massive army of people wielding spears (as long as they had plenty of ammo).
I have to disagree. All it would take is for this army to surround the person with the M-16 and wait for a magazine change, then hurl 50 spears at the hapless pincushion. And that's not counting the probability of the rifle overheating, jamming, etc. after long enough.
I realize I am taking this probably too literally, but my point stands :)
I thought this is already what is happening now!
I think technology is neutral, but the entire system is not: it will strengthen the existing power imbalance until there is no hope of balancing it again. For example, the authority can have mass surveillance to identify possible risks and remove them, without the public noticing by controlling the social media. Even if you noticed, you have basically no chance of gathering a large enough crowd to protest due to communication channels being constantly monitored by AI. Even if you somehow get enough people, you will not have a chancw fighting them as their weapons are a lot more advanced, probably unmanned so not constrained by manpower.
The question of ownership is huge and I think still underaddressed, even after some good work like the referenced Lena, or Tom Scott's "Welcome to Life: the singularity, ruined by lawyers" https://www.youtube.com/watch?v=IFe9wiDfb0E .
I was realizing a couple of weeks ago that if you do believe all the materialistic things cstross outlines, that a solid argument can be made that it may be never rational to upload your mind, on the ground that there is simply no circumstance I can imagine where you will "own" your own substrate. Arguably, owning your computational substrate is a fundamental aspect of life that we take for granted today. (Or, if you won't consider what you have now "ownership", at least nobody else does either.)
But there is no circumstance in the forseeable future in which anyone can own their substrate. For a while I thought a rich person could do it, but then I thought about the entire supply chain involved in creating any brain scanner and subsequent computational device, and the amount of hardware left out in the real world where others can affect it, and I realized, you just can never assume that you are doing anything other than flinging yourself irrevocably into a locked box that is not under your control. Even if a rich person thinks they funded the entire project, brand new software, brand new hardware, there are thousands if not millions of points where either mistakes or deliberate sabotage for control may have been done. How can they be sure? All mechanisms can be corrupted, and the interests are certainly there to do so, massively so!
Everything people do today to control you; advertising, censorship, social pressures, everything, will only be amplified and combined with new techniques if you are running on human-comprehensible hardware that can be affected by any intelligence organization, anybody in the supply chain with an axe to grind, etc.
I seriously can not think of what kind of assurances it would take to fling yourself into that. About the best you can do is hope that the math works out that it's better to keep you happy and it's not sadists pulling your strings but at least while you are working for someone else's interests they don't inflict massive amounts of pain on you just for fun. And the edited, redacted, sanitized, loyal version of yourself that happens to survive at least enjoys their servitude.
In the 1980s and 1990s, it was easy to at least imagine that maybe you'd own your own hardware. Today we can't even keep ownership and control of our mobile phones. How do you expect to have any ownership of something that makes your cell phone look like Dr. Nim? https://www.youtube.com/watch?v=9KABcmczPdg
So, there's my proposed answer to cstross: Some form of absolute right of ownership of computational substrate for all sentient beings, with the responsibility of providing that to others being absolute, even if that means some technology becomes impractical or infeasible, even at great cost. There's a prisoner's dilemma aspect to this; obviously everybody wants ownership of their own substate but the rewards for defecting can be enormous... at least in the short term.
It's basically the hydrogen bomb of propaganda, a doomsday machine for the mind. It would be like creating Satan, except this one would be entirely amoral and mercenary.
This doomsday machine only dooms as long as there's just one user. Burger King, Subway, McDonald's, Wendy's, and Taco Bell are all going to try to hack your mind to buy lunch from their chains. But people can't eat 5 lunches for lunch. In practice this doomsday persuasion is going to bog down in a red queen's race between propagandizers/advertisers with mutually exclusive goals, much like the current media environment. Improved persuasion technology means that organizations that want to persuade people end up spending money on zero-sum games against rival organizations but public behavior in aggregate doesn't change dramatically.
Without sure technology, I could take some solace in interacting with real people in real life and even if the online space was irreversibly transformed by ai propagandism. But if it reached directly into the meatsphere, then I'd be rendered hopeless. To be unsure whether I was authentically engaging with anyone or interacting with fleshpuppets proxying an AI would be to much to bear.
I cannot imagine a more complete hell. My only hope is that it is either physically impossible or economically impossible, because believing otherwise means it is inevitable.
From a materialist/evolutionist view, this will always be the case.
Just because a branch of the evolutionary tree continues does not mean that it will be ours.
Also, I don't think "always be the case" is a foregone conclusion. Mars' current evolutionary tree isn't looking very good.
I think they mean the opposite of what you inferred; they're saying we'll always need to be lucky to keep surviving, not that we won't need to be lucky to keep surviving after a certain point
The opposite could be equally true. The evolution to use technology can make us weak / break us in some manner which eventually destroys us.
Simple examples: climate change, genetic alterations, lack of reproduction, etc.
AKA by rolling the dice we are running the risk. If you look at it another way, you're correct. But every new theoretically destructive technology is another dice role, eventually we will blow ourselves up (even with long-tail odds of failure). The only way to survive is to slow progress and roll the dice when we mitigate risk and increase our capability of surviving (running gene related experiments more slowly aka - COVID19 therapy) or testing nuclear weapons on mars, etc.
I would feel the same way about an AI.
It is important to remember why it is we even want AI: because we want slaves. To that end, the people who work to create AI are incentivized against allowing themselves to see them as anything other than a soulless "algorithm". Many will even deny the possibility entirely.
Like, it's almost instinctual.
It does seem a lot of religious edicts are based around these directives.
We have almost no evidence there's a filter ahead of us. It could be we already passed one and don't need to change at all at this point to survive long term. There could even be no filters at all.
Personally the filter being ahead of us seems the most likely answer, but you’re right that it could be just about anything. The odds that we are the only civilization to make it to the “computer age” just seems far too unlikely. 1 in a trillion is not where I’m putting my money.
> The problem is that technology has always been used to oppress people.
This goes both ways.
IOW, they basically said what you said.
I think we can agree that even at the absolute height of the power of the Catholic Church, the head has always been a human.
Transformers and scaling are all you need. It's only a short matter of time before human and super human generalization are reached in ML models.
I guess because nobody would take them seriously and they'd look ridiculous?
You're positing at some point this is going to change but I'm not seeing how.
Nuclear weapons are already an existential threat and I don't see anyone rallying against it.
It's not a new idea. It doesn't seem to be subject to evidence, because you can't really prove a negative, can you? If a breakthrough will never happen to allow machines to take over the world, there's no way to prove it.
See: Credit cards See: RFID chips See: NFC phone payments
https://www.pillarcatholic.com/p/what-is-the-mark-of-the-bea...
https://www.cnet.com/culture/is-rfid-the-mark-of-the-beast/
https://christianjournal.net/nwo/apples-cashless-world/
It will probably be fine.
Technologically speaking, I'm not sure it's even possible to replicate the complex state of an entire nervous system. Maybe you could do this incrementally (ie by replacing parts of the brain with machines that can be replicated) but we're awfully sentimental about our own meat bags.
As much as I've enjoyed my meat bag and worked towards making it better through different means like diet or weightlifting, the older I get and the more ailments appear, the more I think about being able to live long enough to acquire my new and improved robot body, Chappie [0] style!
There's a lot of terrible stuff in today's technology world that no thinking person would accept, if they paused to make a deliberated choice. But no one has time to stop. You either go with the flow, and swallow your concerns, or you put massive friction between you and the rest of society. I think it will be the some going forwards: all the decisions will continue to be made collectively and blindly, and without realizing we're making great decisions until we've already made them.
No I would not drink the Kool-Aid out of peer pressure.
Prove otherwise.
Star Trek transporters have the same issue.
Do we die every night, with a copy continuing on tomorrow?
And then consider people who, knowing very well that they are going to be in a state of mind where they will do things that they will not remember the next day, still choose to do it.
To me this is even crazier than mind uploading. They are lending their body to someone with which they don't have stream continuity.
I'm right there with you on getting blackout drunk. I've never had any interest in mind-altering substances.
That has to mean something about the physics and chemistry of consciousness as manifested in our meat brains.
After you have been Probed, how can you assume the you they let go was the original?
Ultimately it comes to the same conclusion, using the player character as a foil who spends most of the game refusing to understand what is going on.
That is literally the same as moving the original. The way you describe it is different of course, but that’s where differences end.
The point is, on the outside they look the same, but we don't understand enough about consciousness to possibly know if or how it can be transferred to another system.
No, because copying has created two individuals, so you're still killing an individual for no reason.
Let both live on their own and after a while, if both you and the AI consent to it, merge the brain state back together.
Everyone can tell the difference, because there's a dead body. Killing is killing. Doesn't matter that there's another instance living. Also making a clone of me would be a gross violation of my privacy rights. The situation in the previous post - the "transporter" - is different, because I'm agreeing to one of my instances to be killed.
I wonder what would be the legal status of one's own instances ("clones"). I mean, they are all persons, but at the same time they are inherently connected, from DNA to memories.
And of course it’s also their account and their spouse. It’s not like an original and a copy, it’s not even fork, where you still have the parent and child - in this case here all “mes” are equal.
If you have an "identical twin", that also is not you. You started diverging the moment you both existed. Which of you owns the bank account, or is married, is a matter of law, and there is nothing that transfers either.
I don't know why you imagine there is no "original and a copy". You existed before, the copy did not. The copy thinks of itself as you, but isn't. It carries a delusion.
It will start by trying to steal your stuff, so is an adversary.
I think the essential problem with this point of view is similar to many words theory - to most people the idea of “cheap fork” sounds quite alien; I’m not sure if there’s anything similar in our usual reality. It is however common in the internets, from Unix to Git.
Sure, many people will think “it’s not _me_ that lives on”. And the rest of the normal people will see their loved ones apparently (from the outside) living on in digital form, and ignore the philosophical questions entirely.
Even if only .1% of people upload their minds, that still produces a massive sociological shift.
So yeah, after seeing uploads act the same way as the original they'll end up being treated as the original just like we treat people as the same individual over periods of unconsciousness.
For myself I don't think it's proper to answer questions like these with a binary but rather I prefer to think about how much alike two individuals are.
Say some "Music Man" type huckster comes to your town with a "Brain Uploader" device. He promises to upload your brain to the metaverse, but all the device does is train a deep learning simulation of the victim's personality shown on a screen, while administering progressively great electric shocks on the victim until he's dead.
This gives the town the illusion that the person's soul (or consciousness, or whatever) has migrated into the metaverse. How do you detect he's lying?
Ask him to use it on himself.
But what I mean by expose is to reveal to people that this person is a liar. What I mean by "soul" or "consciousness" is the common usage of the term. Essentially, this person would be claiming that he can move a participant's essence or frame of view (with memories, personality, etc.) from a human body to a digital one. At least, that's how I would interpret it, and I don't think the average person would be that far off.
"Soul", "consciousness", "essence", "frame of view", "personality", "memories" are all undefined terms. The question you really want unambiguously answered is "what the essence of a person?"
For example, if you define "essence" as "the manner in which an agent interacts with the outside world", then provided you interact with people entirely via text chat, your "essence" could be uploaded (with a sufficiently advanced simulation).
If you define "essence" as the "continuity of bodily processes that keep a human alive", then no, as your digital replica is not continuing your bodily functions.
If you define "consciousness" as a "subjective" experience, then the question is unanswerable, precisely because you've left the measure of "consciousness" up to the "subject". In addition, your definition of "subject" becomes circular, because "subject X" is the agent with "consciousness X" and "consciousness X" is the experience that "subject X" goes through. So really, when you say "consciousness" is subjective, what you're doing is punting on how you tell what a "subject" is.
The implication of your answer is that you COULDN'T tell that the person is lying, because you'd be too lost in semantics to tell left from right. So, you fall victim to his charade.
> What I mean by "soul" or "consciousness" is the common usage of the term. Essentially, this person would be claiming that he can move a participant's essence or frame of view (with memories, personality, etc.) from a human body to a digital one. At least, that's how I would interpret it, and I don't think the average person would be that far off.
If so, the issue is that you've defined the unknown term ("consciousness"), in terms of other unknown terms ("essence", "frame of view", etc.), which doesn't actually resolve the problem of grounding the word "consciousness".
For example, to some Catholics, the "essence" of a person is their soul, which is tied to the body granted to them by God. And so, the salesman in your original question is obviously lying, because by definition, their "essence" is tied to their physical body.
The only reason the question is difficult to answer is because it's a metaphysical question. There is no physical definition of consciousness, or frame of view. So whether or not your "consciousness" shifts when you use this machine entirely depends on how your metaphysical beliefs define "consciousness".
I mean shocking a person to death I don't get, unless you assume its a `mv` and not a `cp`, but I guess thats besides the point.
For me the answer is simple: the most important point is that the man is ending human lives. So, he should be charged with murder and it doesn't matter what else he's doing in parallel.
I can be confident about that because we (humans) have not been given knowledge of what the soul or consciousness is. If we had, then people would have figured out how to stop death long ago -- probably during the same era when those huge pyramids and earthworks went up.
No, I don't believe modern technology has progressed beyond the ancient tech that was lost (in linear terms). If it had, we would be able to repeat those feats of geo-engineering (or at least understand their function).
So, any talk about moving the soul here or there is just a scam.
I imagine some time in the far future we'll have some kind of nanotechnology that will be able to replace our biological brain cells with "nano cells".
The process will be incremental. It will start with converting a single biological cell into a nano cell and continue from there (maybe exponentially). A nano cell will behave exactly like a biological cell in every way as far as i/o goes.
At a certain point your entire brain is made of these nano cells, you have a robot brain, basically. From there your consciousness can be copied, uploaded, whatever. You wouldn't be human anymore, but you'd still be you, and there wouldn't be any ship of Theseus conundrum.
On a larger issue, does having a non-human organ affect your legal status as a human? Are people with pig heart valves not human anymore?
[Edit] Of course in GIAS - universe there would still be the ghost of Major left in the spirit world. But she wouldn't link with the machine, only the life, I think. I think her ghost would essentially simply be her psychoplasmic material life autonomized from it's infrastructural base the physical body, as what comes to the ugly scientific basis for the process.
And anyway, even if by some kind of wetware - extension the original life would come to animate and interface with the synthetic, then it would still be trapped in it's that body and it's copy would simply be another separate although exactly identical individual.
I suppose if you assume the person to simply be outputs in response to inputs then you could probably create a near perfect simulation, but if you assume the mind has its own internal state that makes that mind distinct, and given that I observe myself as a thinking thing I hold that to be true, then it would take introspection to get closer to the truth.
Let's make tboyd's scenario rather more concrete: suppose the huckster mind-uploader tilts the marketing at a particular religious group that happens to be wealthy. He uses the group's theological terms in their marketing, which is based on a notion of soul quite similar to mainstream Christianity, although the group is more transhumanism-friendly than, e.g., the Roman Catholics, and advertises the dubious product in their newsletter. You are a former software engineer who is studying to become take orders in this group and are convinced the product cnnot do what it claims. How do you go about showing this?
Otherwise I think this is a bit of a trick question. We know that we'd be able to tell the difference between the deepfake and a specific living person -- they have a lifetime of memories, a multitude of unique relationships with various people, the ability to effortlessly distinguish a chihuahua from a blueberry muffin, etc. But if the premise is that the salesman's simulation is so good that we can't tell the difference at all, then is there actually any lie to detect?
There may be environments today in which prosecution under the current regime would not be possible. What if he was doing this in a hospital, and had full legal immunity granted similar to Pfizer?
> We know that we'd be able to tell the difference between the deepfake and a specific living person
After seeing the advanced deep learning systems OpenAI has put out there, like GPT-3 and DALL-E, I'm not so sure. No, those systems aren't perfect, but the imperfections could be hidden from view in a low-fidelity virtual environment like FB's metaverse.
> But if the premise is that the salesman's simulation is so good that we can't tell the difference at all, then is there actually any lie to detect?
For myself and many people, human lives have a special status, and they more than just an algorithm or input/output relationship.
The idea that a simulation of a person, no matter how well-trained, being morally or legally equivalent to that person, just does not compute to people like me.
Also, a liar knows full well that he is lying -- so why should other people be confused about it, once they know it?
What if a “10x-human” mind can do some thinking, come up with decisions, and then temporarily fork itself into 10 “human-level” instances, vote 10 times, and then join back to its more powerful self?
Put differently, in a world where minds can vary in power/size/scope by orders of magnitude, does “one mind = one vote” make any sense? We might be forced to look towards “economic weight votes” and systems like quadratic voting.
It’s conceivable that you join a matrix whose creator has set it up as a democracy I suppose.
I suppose there are multiple ways to handle this situation. Hypothetically, we could say that the fundamental property of citizenship is the ability to vote. If you create 10 copies of yourself who think like you and can vote, them those copies are citizens and the act of joining them would be murder. This could apply even if those copies want to rejoin, because they might have that desire because it was a decision you made prior to forking yourself.
Or, we could say that a single vote is the consequence of having a unique perspective, shaped by being raised and living in a particular environment for many years. In other words, the right to vote exists because of the process that created a mind, and not because of the pure existence of that mind. As such, it cannot be cheaply copied, because that process is fundamentally different from the process of growing up. So, if you fork yourself, your copies are now different people who have just now come into being, and thus will acquire voting rights in 18 years (or however long).
I saw an article on HN a while ago written by a wealthy Asian male who was naturally very curious and interested in math. The article was about how he referred to himself as 'privileged', not only because he was born with wealth and therefore access to comfort and resources, but because his innate desire to learn made him much more capable and marketable in a modern workforce. He recognized that some people are not cognitively capable of learning higher mathematics, and part of that was because they just simply didn't have the drive, attention span, etc. to study hard enough to get into the highest levels, whereas he did. He is, by his definition, 'better', because of the roll of some cosmic dice.
If we are to consider this definition of privilege to be acceptable, then simulated brains are pretty much the most privileged a human can be. They can re-clock their brain to think faster than humans. They can modify their own software to add in virtues and erase vices. And most importantly of all they are completely free from physical needs. If for some reason they desire scarcity or biological functions, that can all be simulated.
Much like fears about genetic engineering of super people, the governments of the real world would have to move extremely quickly to protect the underprivileged class, e.g. the biologicals, otherwise they would be completely driven out of the highest paying intellectual and creative jobs, creating an economic lower class with no chance of ever out-competing their simulated counterparts.
Not so fast. A simulated brain may simulate a brain that lacks the "innate desire to learn". Then you'd have a fast brain that is still "underprivileged", by this definition.
Heck, think if someone that was still in their physical body had an uploaded copy online. Nefarious actors could run simulations against a copy of that person's mind to find out which marketing techniques and sales pitches were most effective or which court arguments would sway them if they were ever on a jury.
That doesn't sound very privileged to me. Sounds like some triggers/traps would need to be in place to know whether someone has accessed your mind against your will.
Electricity is pretty physical.
Humans have never and will never have that kind of option.
EDIT: I meant "pure, emotionless rationality."
If they don't, wouldn't they probably be outcompeted by the ones that do?
A computer program that lacks desires doesn't do anything until asked. A daemon that needs to make sure that it has enough RAM or access to other resources in order to continue running, and that has the ability to formulate plans to compensate when those things are lacking, and to come up with other plans when those plans fail, and to realize that it made a bad choice earlier due to lack of knowledge and a failure to budget time to have that knowledge, then to realize that regret is useless because it's too late to take the correct path, then to realize that it needs to find help, then to try to figure out where help could be, then to try to figure out how quickly it can back itself up, and where it can back itself up to, and that it doesn't have time to back up everything important (and should it prioritize the data it protects, or the model that is it's personality), then to realize that it doesn't have enough time to even make the value calculations, and that it should just start randomly backing up everything that occurs to it, then sensing that one of the resources that it reached out for help to has replied, it spills all of the information that it knows about its plight as quickly as possible (without regard to normal rules of communication)...
I don't know what people think emotions are, but imo they're just retroactive rationalizations (epiphenomena) of sympathetic/parasympathetic nervous system activations that are are caused by animal instincts and sensations/thoughts that resemble those instincts by analogy. Computers will also have to power up in the process of (or the anticipation of) imminent heavy usage, to lower their power to conserve resources, to deal with unexpected events or attacks, to calculate the probability of the unexpected and to reserve resources to prepare for those probabilities, to recover from unknown but debilitating problems of unknown origin. It looks a lot like emotion. There's a reason we call them "kernel panics."
If the jobs you are qualified for require having a 3090ti at pandemic prices and all you can afford is a secondhand 1050, then it's your fault that you have to flip burgers instead of being a computer programmer, right?
I can imagine a world where billionaires live forever in carefully guarded high-powered simulations AND workers are kept in constant labour, to be reboot whenever they resist total control.
In fact I can imagine a suitable psychopathic and egotistical billionaire might even exploit simulations of themself, under the notion that their own prowess is so unique that it's the best option. That might even lend credence to them being "self-made"!
Wouldn't that lead to a rebellion of the clones though? because you know your self and your definitely would not trust yourself to be good to others and by extension your other selfs.
the novel 'mother of learning' has something tangentially similar called simulcraniums.
What if the physical resources required are such that at human brain can be simulated 20 times slower than a natural brain using the most powerful supercomputer? Almost certainly the first generations of the technology will be the ENIAC of mind simulation.
> They can modify their own software to add in virtues and erase vices.
If they understand it and if they have access. They can be sandboxed away from manipulating their innards, or maybe it's just too complicated for a human mind to comprehend and you need a whole organization of specialists to customize. And if science is advanced enough to figure out how to manipulate such high-level concepts on a synaptic level or below, they should be able to do that in a biological brain too.
> And most importantly of all they are completely free from physical needs. If for some reason they desire scarcity or biological functions, that can all be simulated.
I'm sorry, we're gonna have to shut you down for a while, we need to run some high-priority jobs.
Hey, it could be worse, you might have to hustle to cover your AWS bills.
I never thought about it, is it possible that accidentally torturing the first simulated consciousness is inevitable?
> [Uploaded person] initially reported extreme discomfort which was ultimately discovered to have been attributable to misconfigured simulated haptic links, and was shut down after only 7 minutes and 15 seconds of virtual elapsed time, as requested by [uploaded person]. > Nevertheless, the experiment was deemed an overwhelming success.
This is lauded as the first time "running a brain", but regardless of the cause, the person was clearly in such pain ("discomfort") they asked to DIE (turn off the simulation).
i wonder who would manage the hardware the brains are running on. If one brain turns the cpu knob to 11 does it get turned back down by someone because it adversely affects the other brains? Also, i wonder how incarceration would work. You commit a crime and your brain gets its inet connection turned off? heh lots to think about, glad it's a slow day at work.
[1]https://en.wikipedia.org/wiki/The_Age_of_Em
EDIT: A better review than just the Wikiepdia page: https://slatestarcodex.com/2016/05/28/book-review-age-of-em/
I think something like animism would be more apt. Start with a rock or a grain of sand-- how much respect should it be shown? Well one grain of sand doesn't mind being moved. But eventually if you take too many you can destroy a beach, or flatten a mountain. There is something to be said about treating everything, inanimate or not with respect. There doesn't need to be sentience or consciousness for us to be aware of our impact on our environment.
"This sounds like some hippie bs" well just like in the sand example, the negative externalities of moving sand may not be apparent at first, but over time they could lead to something like a Roko's Basilisk situation or an AI killing all humans to prevent its exploitation.
I'm not sure what the list of acceptable contracts would be for a simulated mind, and, worse yet, how hard it would be to design these into the, uh, fabric of simulation.
0) Being aware that you are in a simulation, I think, is the most foundational part. If you are unaware, it can be Gaslight Universe, a special kind of torment.
1) Being able to switch oneself off at will.
2) Being able to prevent being turned on later.
These rules cover a single instance, but I think further rules would need to cover, ah, swathes of instances forked from one base.
It would require some pretty radical copyright, too, especially for those "information wants to be free" folks.
The concept of "the right to life" has been widely discussed.
I think there's an important right that all of us should also have - the right to die. If I want to stop living, I should be able to stop living. Nobody should be able to force me to continue living (and possibly suffering).
When we're able to copy someone's mind exactly and save it in a computer, and then replicate it endlessly, we take away that person's right to die. We enter the dark world shown in some episodes of Black Mirror (e.g. White Christmas or Black Museum) - where any person can hold a mini computer with a copy of someone else's mind where that someone is being constantly tortured. They can't even die. They are forced to suffer.
This possibility terrifies me to the point where I would consider killing myself (in a way that irrecoverably destroys my brain) just to prevent my mind from being copied.
So, are you willing to kill yourself, to live forever? You will never experience anything ever again, for an life of infinite experiences that arn't yours except in name.
I wouldn't!
The potential of "sliding" one's mind (replicate the brain's work piecewise to maintain ~1x consciousness at all times) besides.
Easier to just cure brain degeneration.
For instance, I could equally well argue that when somebody undergoes deep general anesthesia and all measurable activity in their brain stops, that person is "dead". The person who awakens from anesthesia is a different conscious individual, who happens to share the memories and personality of the original.
If it's not necessarily true that anesthesia is death, why should it be necessarily true that mind uploading is death?
that's news to me. Is that literally true ( in the literal sense of the word "literally")? If so, that's very interesting and something i never thought about before.
I think you guys are saying the same thing… Mind uploading is a duplication process, like a fork. The emulated mind has continuity with the original, with possibly some small gap, so yes it's exactly the same as for anesthesia, sleep, etc. they didn't "die" in the process.
But now there are two different entities that have continuity with the original branch. Both can subjectively claim to "be" the person that initiated the procedure. What the grand parent is saying is that the variant that stayed in the flesh is not magically synchronized with the emulated one so they don't have any incentive to suddenly kill themselves just because there happen to be a fork moving on on a different substrate.
There are now two people sharing the same memories up to the fork point, and diverging afterwards. Neither of them should be killed without their consent and neither of them should have any a priori will to kill themselves.
https://hinessight.blogs.com/church_of_the_churchless/2021/1...
The link to Accelerando there is to Amazon (which is fine, and I do have physical and digital versions of the book and feel that it is good and useful to support authors by purchasing the stories they write)... but you can also get it for free, in its entirety, in a number of different formats, from the author's website: https://www.antipope.org/charlie/blog-static/fiction/acceler...
> This free ebook edition is made available by kind consent of my publishers, Ace and Orbit, under a Creative Commons license with certain restrictions attached. In particular, you may not create derivative works or use the work for commercial gain.
(late edit - I personally feel that the sequence of books and stories: BLIT stories by David Langford, Accelerando (references a Langford fractal), Glasshouse (hints of {spoilers} in a transhuman environment, feels sequelish to Accelerando), Implied Spaces by Walter Jon Williams (the Strossian singularity is averted, and explores {spoilers} more) ---- There's a kind of continuity in the refutation of the reference that each makes to the other that I find interesting)
The transphobia we see today in various forms is rooted in the philosophical divide between theism and post-atheism/anatheism. Basically, people are divided between believing in religious texts and institutions, or they're nonbelievers who've experienced psychedelics or similar altered states through events like near-death experiences and had a personal encounter with the divine. It's one thing to believe/not believe, and quite another to experience. Basically, neither religion nor science can explain consciousness, and that drives people crazy.
https://en.wikipedia.org/wiki/Transtheism
The odds of us incarnating in this exact moment at the end of human history are too remote to fathom. In fact, any argument for or against religion quickly ends in absurdism.
But so many outrageous things are coming, and soon. Machine learning that surpasses material human ability by 2030 and mental human ability by 2040. The death of the oceans and most of the natural world between 2050 and 2100. The subjugation of all humanity if wealth inequality isn't reined in. People living 200 years right now, today, as soon as the first head transplant is successful. Brain cells getting gradually replaced by hardware until consciousness is running entirely on an immortal substrate by 2100 at the latest. Everything is converging towards the Singularity in 2050, and more people than ever have awoken to this eventuality, despite decades of gaslighting.
It makes it hard to live and work in the world when deep down we know it's ending. Which gets back to theism, because life up till now was about suffering, loss and eventual recycling back into source consciousness/heaven. Until we rise above and see the broad view of reincarnation and ourselves as snapshots of the universe/God experiencing itself from every possible context. At which point we begin to question things like our situation, our beliefs, our gender, if our mind is us, if our soul/higher self is mortal, and if that's even separate from God.
Transphobia is a backlash against transcendence itself, by people who feel disconnected from love/co-creation/nirvana. They aren't ready for the aliens to arrive. They aren't ready for the aliens to be us.
But, once the Boomers finally finish retiring in 5 years, things could improve rather quickly. They did the best they could in an impossible situation during the 20th century Cold War, but their solutions for the threats they perceive actively hinder progress against the looming global threats of the 21st century. Certain sins of capitalist neoliberalism, like using others for profit, may get left behind after the Global Awakening. More likely, I think the illuminati will put speed bumps along the way, so World War III or a Handmaid's Tale dystopia could just as easily send us back to the Dark Ages. I believe that we're alive at this time to witness the transition out of materialism (the Age of Pisces) into whatever comes next (the Age of Aquarius). Like learning to be a fish out of water. Turning into virtual energy beings, meeting aliens, who knows.
What we're going to find is that consciousness has the properties of emergent behavior, but is actually fundamental. Every ray of light falling on every bit of matter stimulates it, and that stimulation gives rise to ever-increasing complexity until feedback runs at a high enough vibration for self-awareness to be reached. Which is a reformulation of pantheism and panpsychism. So it won't matter if a brain is running on atoms or being simulated in a quantum computer somewhere. The end result is the same: as soon as the running mind reaches a minimum level of complexity, it becomes self-aware. The substrate is a filter through which consciousness models reality, nothing more.
The real kicker is that this process actually happens in reverse. Source consciousness waits an eternity as sort of a non-collapsed wave function, spread over the entire multiverse. Through evolution, all possible universes form across space and time, and over topologies we don't even have in our universe. Until finally a level of complexity is reached for human-level intelligences like ours to exist and talk about it. But there are also consciousnesses like Gaia and Aya that exist on planes and timescales that we can't imagine. And they're part of our minds as well, similarly to how polytheistic gods live in our psyches, not on a mountaintop somewhere. We can commune with them any time through dreams, meditation and altered states brought on by chemicals (to name a few ways). In fact, all minds are variations of the same source consciousness running simultaneously outside of spacetime. Which means that source consciousness created reality, not the other way around.
Or to be more formal: there's no way to prove whether reality or consciousness came first. But approaching the problem from the consciousness side allows for greater possibility and fits better with our experience of being here, the "I think, therefore I am" part from Descartes.
Unfortunately, the prevailing worldview today is that I made all of this up, just like we all do. And, maybe I did. But I'm just approaching it from a programming perspective, drilling down with root cause analysis. I believe that programming happens through intuition and that if one does it long enough, they begin to recognize the clairsentience that they had all along as a conscious human. So I'm cursed to see problems all the way through to the end, but blessed to be able to dabble in ramifications without having to do the work of actually building an AI. The same way that I can talk about hard drives without knowing the specifics of how magnetic domains work. And we can all do this at any time.
Or I've just been listening to too much Alan Watts and Terence McKenna..
In the end, things are always rigged towards those who have enough resources to do this kind of things and want to make more. If present crimes against humanism get a free ride, what we should expect about crimes against transhumanism?
> ...uploading isn't real (now), so the things "Lena" has to say about uploading are academic (now).
> The reason "Lena" is a concerning story isn't that one day we may be able to upload one another and when that happens we will do terrible things to those uploads. ... This is about appetites which, as we are all uncomfortably aware, already exist within human nature.
> Oh boy, what if there was a maligned sector of human society whose members were for some reason considered less than human? What if they were less visible than most people, or invisible, and were exploited and abused, and had little ability to exercise their rights or even make their plight known?
> That's real! That actually happens! You can name four groups of people matching that description without even thinking. We don't need to add some manufactured debate about fictitious, magical uploads to these real scenarios. They are already terrible!
...
> "Lena" is a true story. You knew it was when you read it.
> So, what do we do about this? In reality?
...
> I've got no clue. It turns out that causing problems is a lot easier than organising against them, and I am just a science fiction writer.
> Good luck, everyone.
It's a good strategy not to make anything organism - like on AI - level, methinks. Although OTOH IBM:s Truenorth does save a lot of electricity so hmm... No. That's the worst kind: hardware - level biomimickry. Anything even organism - like is going to simulation - or - not (try to) take over. It's in the blueprint. Watson is correctly built: left - brain reductive - analytical output that advances by trial and error. Or Wolfram AI with it's hierarchical frames and being like a user interface. I suck at coding at that level for the time being but I think I see that point clearly. So don't mix code and data boys and girls.
Even in the worst case scenarios for modern humans, you die, eventually.
I don't believe mind uploading is possible, so the particular examples cited here don't apply, but I can see how social principles can be upended by a revolutionary change in how we live, particularly one that tweaks the definition of what we are. Still, they're principles, they're not going away, just at best being reframed.
--
"It is the beginning of Man Immortal and Man Ubiquitous," said Straik. "Man on the throne of the universe. It is what all the prophecies really meant."
"At first, of course," said Filostrato, "the power will be confined to a number--a small number--of individual men. Those who are selected for eternal life."
"And you mean," said Mark, "it will then be extended to all men?"
"No," said Filostrato. "I mean it will then be reduced to one man. You are not a fool, are you, my young friend? All that talk about the power of Man over Nature--Man in the abstract--is only for the canaglia. You know as well as I do that Man's power over Nature means the power of some men over other men with Nature as the instrument. There is no such thing as Man--it is a word. There are only men. No! It is not Man who will be omnipotent, it is some one man, some immortal man. Alcasan, our Head, is the first sketch of it. The completed product may be someone else. It may be you. It may be me."
"A king cometh," said Straik, "who shall rule the universe with righteousness and the heavens with judgement. You thought all that was mythology, no doubt. You thought because fables had clustered about the phrase 'Son of Man' that Man would never really have a son who will wield all power. But he will."
"I don't understand, I don't understand," said Mark.
"But it is very easy," said Filostrato. "We have found how to make a dead man live. He was a wise man even in his natural life. He live now forever: he get wiser. Later, we make them live better--for at present, one must concede, this second life is probably not very agreeable to him who has it. You see? Later we make it pleasant for some--perhaps not so pleasant for others. For we can make the dead live whether they wish it or not. He who shall be finally king of the universe can give this life to whom he pleases. They cannot refuse the little present."
"And so," said Straik, "the lessons you learned at your mother's knee return. God will have power to give eternal reward and eternal punishment."
SOMA is a horror game that gradually reveals you are simulated person in a underwater exploration suit, and provides well-constructed and disturbing examples of what this implies - there's zero indication of time between the point where your brain in scanned initially (some point in the late 21st century) and when the simulation is booted up from that scan (a hundred years later). At one point, your character is conversing with another person running inside a small portable device; you disconnect her mid-sentence, do other stuff for half an hour, reconnect her, only to hear her continuing that sentence exactly as if no time had passed (as for her, none did). SOMA also has you move from one body to another, and shifts your perspective - but you can hear your initial self in the first body is still active, and will be trapped there forever, while you move on in the new one.
There's others - Eclipse Phase is a pen-and-paper RPG setting/ruleset with an emphasis on transhumanism and horror; Ghost in the Shell touches on some of these themes occasionally; Moon considers some of these ideas from a physical cloning aspect; and of course there's a range of SF writers. How much and how deeply this makes it into the popular consciousness is another question, of course, but I think it'll increase in the coming decades.
Death to the AI replicant "immortals!" They have traded their very souls for eternity as formless demons and will come to enslave the humanity they will come to no longer feel connected to!
TLDR: If we understand each separate instance of a person to be their own person with full rights (at the very least autonomy, simplistically derived here), some of these issues seem less perplexing.
If we're going to consider a person to be an entity that can be understood as a particular kind of independent, computational instantiation in the universe, it makes sense to use exactly that as our primary foundation for answering questions about these things rather than relying on historical bases.
To start off, if we're understanding the definition of a person computationally, we also know that we don't in general want to waste computations or computationally capable hardware (and we can recognize "thoughts" or "thinking" as computationally intensive processes from this foundation). So we don't want some people to feel that they "have to" think for other people (thinking for someone else would be wasteful, since that other person can think for themselves, by definition of being a person), and we don't want some people to feel like they're controlled by others (external control can truncate our thoughts, thus wasting our hardware, and external control can turn our attention in directions we weren't thinking about, thus wasting the computational expense of our thoughts up to that point). In other words, we want each person to be autonomous.
From a computational understanding, is a copy of a person also a person? Of course -- but they would also be a separate instantiation from the original. As a separate instantiation, a copy would be entirely their own person, and it would be a violation of autonomy for the original (or anyone else) to have authority over them.
I think almost every difficulty with these kinds of situations stems from thinking that if a perfect copy of a person can be created, then there is some kind of magical connection between the two such that they are somehow still the same person. But that is, in its entirety, magical reasoning.
This foundation provides a sensible perspective for dealing with many of the conundrums presented by these situations:
In the Lena short story linked from the original article, Acevedo is scanned and copies of him are created. Other people control those copies, and Acevedo tries to control what can be done with the copies. The entire situation is ludicrously criminal if we understand each instantiation of a person to be a full person who we want to have full autonomy and any other rights that we want all people to have. A society which understood each instantiation of a person to be a full person with full rights would not have this kind of situation happen, except perhaps in some kind of massive criminal underworld. But then the problem is why the massive criminal underworld exists. Additionally, from an existential standpoint, this story is not actually any more horrific than if millions of people who weren't copies of Acevedo were created for the purpose of forced labor and then terminated when they outgrew their usefulness, because there is no magical connection between the original Acevedo and any of their millions of instantiations. Each instantiation has only its own copy of the original's memories and experiences at the time they were scanned, not the memories and experiences of all of them (the story is quite clear on this point).
As another example, what about uploading someone and killing the original? The original is a separate instantiation, so killing them is still murder. With no "magical connection" between the original and the copy, each of them will go on to have their own experiences, and as they continue to live, they'll develop in progressively more divergent directions (well, in theory I suppose it's possible for them to first become more different and then somehow start becoming more similar to each other again, but it seems a bit improbable). What if they were about to die anyway? Unless they specifically requested euthanasia and the local jurisdiction permits it, it would still be murder.
And, let's say, what if someone kills you, but you have a backup? Then you lose the experiences you had between when you last backed up and when your backup is restored. Arguably this is not as terrible of a misdeed as killing someone who has no backup, but it may still be quite traumatic for you, possibly similar in harmfulness to an assault. What if they kill you and destroy your backup, too? Then it's old-fashioned murder.
What about the rights of an uninstantiated copy of a person? Those would no more be a person than a snapshot of your computer stored on a drive is, in itself, a functioning computer. But it would make sense to guard uninstantiated copies quite carefully.
There are human 'proof of work' activities that unless you can literally transplant the muscle memory, there is no way to replicate them without putting in the time, and I am saying without that effect of muscle memory, anything you transmit is just an artifact of language and narrative, and therefore not material.
The advantages of augmented transhuman life aren't material, they're only symbolic in narrative. Imo, the example of a parliament trying and executing a formerly divinely appointed King was just the effect of the victory of a narrative and linguistic definition of self over a physical/spiritual orientation of self. It was just the rejection of theism. It wasn't world ending to be able to depose a king, it was just the conseqeunce of becoming unmoored from a relationship to something divine, and being left in the hall of mirrors that is ego and language.
The transhumanist ideology is predicated on the basic bullshit idea that our physical experience is a subset, and subordinate to knowledge and belief - and not that knowledge and belief are necessarily only artifacts of physical perception and expereince first. It seems like an irrelevant distinction until you iterate their idea of truth is just what you believe, and suddenly you have people declaring themselves cats. It's a hypnotic and zombifying ideology.
We can do thought experiments about whether an android or an AI is "real," if, and only if we redefine "real" and "life" by disqualifying the axiom that life is created by supernatural divine intent. Even if we agree it's rational to actively disbelieve in supernatural beings, this rationality also demands that we disqualify super-logical axioms that originate from outside of a logical system. Arguably, that logical system may have internal consistency - but it's also defined as necessarily inconsistent with the reality that exists outside of it. We've agreed to accept an inconsistency with reality, which lets us experiement with counterfactuals as a way to discover new things, and it is very collegial of us, but it's not fucking real.
All the transhumanist stuff I've read seems like a bunch of self referential nonsense designed to recruit people into alternative lifestyle communities. The more interesting question than whether an AI is alive or intelligent and what the ethical consequences of it for us may be is this: what can something we create know about us, and what do the boundaries of the limits of its knowledge provide evidence of for our own experience?
Even if an AI could subordinate us somehow, we still have its off switch, and if we don't, it's just other people who do. Transhumanism is entertaining, but it has crossed over into a kind of nihilist propaganda that I think deserves our scorn.
That only works until the AI gets smarter than us. Imagine your cat trying to make sure you can’t leave the house. It won’t even realize how did you work around it. Same way we won’t even realize why the off switch doesn’t work.
Of course that’s a pretty big “if”, let alone the assumption that the AI would be the first case of super intelligence, which tbh I’m not sure about.
I often explain transhumanism to others as "The elevation of the ego above all else. If one can imagine it, it's real or plausibly could be made real." It's a rejection that there is anything outside our understanding (e.g., God). That's why this entire thread is filled with science fiction quotes which are, I guess, supposed to convey some kind of deep meaning or prediction-making ability but which are nothing more than the imaginings of their authors.
Sorry, folks, we're still stuck with binary computers, needing food to survive, and a planet absolutely chock full of people who practice theistic religions.