Hypothetical: If the only way we could build AGI would be to somehow read everyone's brain at least once, would it be worth just ignoring everyone's wish regarding privacy one time to suck up this data and have AGI moving forward?
But it’s not at all a similar dilemma to “should we allow the IP empire-building of the 1900’s to claim ownership over the concept of learning from copyrighted material”.
As I wrote it out, I didn't know what I thought either.
But now some sleep later, I feel like the answer is pretty clearly "No, not worth it", at least from myself.
Our exclusive control over access to our mind, is our essential form of self-determination, and what it means to be an individual in society. Cross that boundary (forcefully no less) and it's probably one of the worst ways you could violate a human.
Besides, I'm personally not hugely into the whole "aggregate benefits could outweigh individual harms" mindset utilitarians tend to employ, and feels like it misses thinking about the humans involved.
Anyways, sorry if the question upset some people, it wasn't meant to paint any specific picture but a thought experiment more or less, as we inch closer to scarier stuff being possible.
Sure, if we all get a stake of ownership in it.
If some private company is going to be the main beneficiary, no, and hell no.
But we do, in the sense that benefits flow to the prompter, not the AI developers. The person comes with a problem, AI generates responses, they stand to benefit because it was their problem, the AI provider makes cents per million tokens.
AI benefits follow the owners of problems. That person might have started a projct or taken a turn in their life as a result, the benefit is unquantifiable.
LLMs are like Linux, they empower everyone, and benefits are tied to usage not development.
The price will be ratcheted up, such that the majority of the economic surplus will go to the owner of AGI - with pricing tiered to the query you're asking it. The more economic utility the user will derive from making the query, the more the AGI's owner will charge them.
If it matched human intellectual productivity capacity, that ensures that human intelligence will no longer get you more money than it takes to run some GPUs, so it would presumably become optional.
Would you trust a businessman on that?
So in this scenario I could see it become necessary from a military perspective.
The fact that this is an active question is depressing.
The suspicion that, if it were possible, some tech bro would absolutely do it (and smugly justify it to themselves using Rokkos Basalisk or something) makes me actually angry.
I get that you're just asking a hypothetical. If I asked "Hypothetical: what if we just killed all the technologists" you'd rightly see me as a horrible person.
Damn. This site and its people. What an experience.
I say it is no different than the people who are claiming they don't care. They absolutely do care, but at this point, saying "no" makes you the odd one with obviously something to hide, so they do this from a place of duress.
Unfortunately, I feel we are not too far from people finally snapping and going off the deep end because it's so pervasive and in-your-face that there is seemingly no escape left.
tbh human rights are all an illusion especially if you are at the bottom of society like me. no way I will survive so if a part of me survives as training data I guess better than nothing?
imo the only way this could happen is a global collaboration without telling anyone. the AGI would know everything about all humans but its existence has to be kept a secret at least for the first n generations so it will lead to life being gameified without anyone knowing it will be eugenics but on a global scale
so many will be culled but the AGI would know how to make it look normal to prevent resistance from forming a war here a war there, law passed here etc so copyright being ignored kind of makes sense