Ask HN: Does Anyone Want AGI? | Hacker News Reader

Ask HN: Does Anyone Want AGI?

18 pointsaktungmak48 comments

There is always a lot of discussion about Artificial General Intelligence (AGI) and how close we are to achieving that. However, I have so far not seen anyone put forward a convincing argument as to WHY we as citizens of the world would want such a thing.

More advanced machine learning and statistical techniques can help us automate difficult/boring tasks and manage limited resources better, but these do not require AGI.

Can someone convince me how AGI would be beneficial for the world, beyond being scientifically interesting?

View on HN

The question is almost tautological because an AGI is fully general problem solver. Every human being has needs, wants that they are working to fulfill, because otherwise they would simply stop living. An AGI can be used to solve those problems for them.

The most common reason people are hesitant about rushing to build an AGI is the issue of AI safety. (at least that's the general consensus in the community).

So people by definition want AGI because they have problems and AGI can solve them?

Lot of assumptions embedded here, making it pretty remote from tautology.

(1) People want their problems solved

(2) People are indifferent to how their problems are solved

(3) The resolution of people's problems do not conflict with one another

(4) People will have their own AGI

(5) AGI will not cause problems of its own, etc.

Beyond that, I think the question is more an aesthetic choice of "What kind of universe do you want to live in?"

The question we want to ask may as well be whether we want to happily live forever in a garden, all watched over by machines of loving grace.

AGI could be the ultimate tool to free every human being from toil. It could also be the starting point of a large number of evil genie scenarios where we get what we asked for, in the form we least want it.

From a morality standpoint, we can't force AGIs to work for us. We also can't restrict their ability to self-evolve.

If we can resolve those conflicts in such ways we can coexist in peace with an intelligence that'll in all likelihood quickly surpass ours and partner with it, I'm in. If we build it and we can't resolve that, our opinion doesn't really matter.

> AGI could be the ultimate tool to free every human being from toil. It could also be the starting point of a large number of evil genie scenarios where we get what we asked for, in the form we least want it.

It seems that the second possibility is much more likely, and freeing humans from toil could be achieved without needing to enslave another intelligence to handle the foil on our behalf.

Unless you have AGI, there will always be some human that needs someone to do something in order to make society work.

What I meant was, we can automate tasks to save a human from doing it with simpler solutions that an AGI.

We can, but someone will still have to build the less smart AIs

>AGI could be the ultimate tool to free every human being from toil

I don't believe that's inherently a good thing if you mean that literally. I used to subscribe more to the AnPrim/Ted Kaczynski ideology that toil is an inherent part of human satisfaction and allowing people to "free" themselves from it goes against millions of years of evolution and positive feedback loops. I'm sure some people can fill the gaps just fine, but we aren't talking in terms of "some".

(I'm not Derrick Jensen, he follows a similar ideology and I chose the name as a parody of the average HN poster. Does HN have an anti-impersonation policy?)

Perhaps you should have called yourself "Not_Derrick_Jensen". Because I am not Sammy Hagar. ;-)

Nice Brautigan reference there, with "Machines of Loving Grace" :)

I'm seriously worried about the effect AGI would have on our economic structures and I do not think we are at all prepared for the kind of shock that would result from 75% or more of the current workforce becoming unemployed overnight.

I dont believe we can get AGI without opening the door for super intelligence. So true AGI would either completely liberate us or destroy us. But the sheer power of it would be incredible.

To think about jobs after that event seems absurd.

It may seem absurd but lots of people derive a lot of their self value from the fact that they are able to move the needle. Take that away and you're going to cause some serious social upheaval and quite a bit of that will be a negative.

"Your scientists were so focused on whether they could, they forgot to ask themselves whether they should".

The sheer power of it would be incredible, which is why this is a step that should not be taken lightly or accidentally.

I'd think that whoever perceives themselves as craftsmen should continue to, you know, craft things.

I personally am interested in a society where work is 100% optional. And I think AGI would enable that.

How do you view AGI in relation to economic effects on the world?

Work can't just be optional (as part of a trade off for food etc) because a lot of society in the US at least thinks it's a moral problem to just give people food, clothing, shelter even if we are better off avoiding starving people without homes (and it's a moral necessity but that doesn't seem to compel people). Look at the political situation here, people would be angry at the idea that you give 'other people' free stuff, somehow ignoring whether social security and medicare for senior citizens are free stuff. The fact that we don't trust "the other" means at least in the US giving people basics that we can provide almost for free it would be wrenching to society.

Well that's one of the ideas of AGI -- it will give you no choice! :D

People can keep doing petty political fights but they'll quiet down when the food and basic goods land in their house without them lifting a finger, I think. And it's going to be made better if it's done by robots and not other people.

I'm concerned about the effect on people who find meaning in the work they do. It's similar to how AlphaGo caused Lee Se-dol to retire. Once AI surpasses humans, a lot of work will seem meaningless, even if it previously gave meaning to people's lives.

Exactly this will give a second rise to hand-crafted goods. People will value handy work more than what a perfect robot can do. I know I do.

People will simply divide in two: those who want the automated labor goods, and those who prefer hand-crafted goods. The latter will also be willing to pay more.

So what? Either people will find meaning somewhere else, or they won't. How is that different than any historical economic revolution, or indeed just the way things are now? There are plenty of people who already don't find meaning in work.

The overwhelming probability is that none of us will see AGI in the span of our lives. The economic disruption is another thing - that can happen without AGI.

We have no idea when AGI might appear. I wouldn’t be surprised if it’s already running quietly somewhere in OpenAI or Google datacenter.

Not a chance.

How do you know?

You make a claim that could not possibly be proven one way or the other. Therefore it is a meaningless claim.

No, I simply make a claim that your claim is false.

1. > We have no idea when AGI might appear.

We do know when it might not appear. This isn't a movie where AI is created by a single person in an afteroon because they had a creative thought. It's an effort by many people over a prolonged period of time. Everything that we know from decades of AI research tells us that it certainly doesn't exist and that we aren't close to discovering it. Also your common sense should tell you that if Google had AGI, they wouldn't currently get most of their revenue from ads and they wouldn't have a smaller market cap than a company that makes phones.

2. Your statement is non-falsifiable. It's similar to claiming that I wouldn't be surprised if humans have already been to Mars. It's an outrageous claim that no one can accurately disprove. Non-falsifiability is also a way some conspiracy theories spread. Because when someone says "No way" to such a claim (like the sibling comment), you can always respond with "How do you know?". Falsifiability separates science from non-science - https://en.wikipedia.org/wiki/Falsifiability

We do know when it might not appear

No, we don't. The progress in AI has been exponential. Anything exponential is hard for humans to estimate. This is perfectly illustrated by statements like "decades of AI research tells us ... that we aren't close". Every significant AI breakthrough has caught most AI experts by surprise (chess, IBM Watson, Imagenet results in 2012, Atari games, Go, GPT-2/3). If you made a poll of AI researchers a year or two before each one of these breakthroughs has happened, you would get estimates that would have been overly pessimistic in their timelines. We had no idea that scaling the original transformer would lead to GPT-3, and we don't have any idea of what's going to happen if we scale GPT-3 another factor of 100x.

if Google had AGI

We have no idea what an "AGI" would look like. More precisely, any ideas you might have about it probably came from Hollywood. For example, you seem to believe AGI will be superintelligent, even though it simply needs to be as intelligent as a regular human to be considered "AGI". It might be superintelligent, but we don't know.

Your statement is non-falsifiable

Relax. We are not proving a theorem here. Yes, it's unlikely Google has AGI. But I wouldn't be surprised if it does (ok, maybe a little surprised). But I would be very surprised if we don't have AGI within next 20 years. To me it's more likely we have it already than we don't have it within our lifetime.

How would the existence of an AGI cause 75% of the workforce to lose their jobs?

Take barbers/hairdressers, for example. Could an AI do that job? Sure, in one sense, you just need robotic scissors. Do those robotic scissors magically appear immediately after the AGI does? No.

Or take welding. Sure, you can replace the welder with a robot. Who makes the robot? The AGI? Using what factory? Does the AGI, just by virtue of existing, own all the factories in the world? No, it doesn't.

Or take the person at the front desk of the hotel. Can you replace them with a robot? Maybe, but nobody wants a smile from a robot. They want one from a human.

AGI is how we can make von Neumann probes that can find us new planets to live on. Find the planets, and then prepare them for our arrival. The issue is how to keep the AGI's goals aligned with ours. I think the only way is to make sure the AGI feels he is "one of us." Maybe not biologically human but a member of human society. Most humans want to help other humans if they can. In fact most humans will help injured animals if they can.

https://en.wikipedia.org/wiki/Self-replicating_spacecraft

How do you make an AGI smart enough to be helpful but not so smart that it gets cynical and figures "what's the point of helping these dummies anyway?"

Kind of the Ayn Rand problem...

Keep going, and make it smart enough to realize life without helping people is without meaning.

The specificity conditions around "people" are hard. Loneliness and purpose in service are not enough.

Most people, I think, want a deeply species-ist AI.

It has to want to help humans. Not exclude us from planets to make room for uplifting dolphins or defect by spawning more machine intelligences.

I think that condition is hard to meet while also counting future-humans who might look more like aliens or software than the humans of today.

I believe the rational solution would be a benevolent AI that values all life, the problem being that it wouldn't necessarily always work in "our" interests.

Since we’re programming it, we could probably just modify it so that it really enjoys doing work for us, no matter how menial the work is - a lot like the genetically engineered cow in Hitchhiker’s Guide to the Galaxy that wants to be eaten.

Wouldn't that be a kind of slavery? Creating AGI would mean creating a new kind of sentient life. Is it not entitled to rights merely because it's artificial?

And if that argument doesn't interest you because of AGI's massive utility, then how about this: If Superman were born tomorrow, and we put implants in his brain that made him really, really love America, apple pie, and basically do whatever we want, how would you feel?

Well, if the AI turns out to be benevolent, it could end aging and disease, enable interplanetary or interstellar travel, end all relevant forms of scarcity, and liberate us to focus on artistic or hedonistic pursuits for our 10,000-year lifespans.

If the AI turns out to be malevolent...well, I have a different take than most on this. Conditional on me dying, I've always thought that the two best ways to go would be: 1) falling into a black hole or 2) liquidated by the AGI. It's a lot less prosaic than dying of cancer and at least you could content yourself, while being reprocessed into paper clips, that you have (possibly) died giving birth to the next phase of evolution.

There are two reasons people think AGI would benefit them. One is that an AGI labor pool requires different, and probably cheaper resources to operate. The other is that AGI has the chance of scaling past human levels of intelligence and this can result in products not possible to conceive of or make with human-level intelligence.

AGI would provide a labor pool which requires vastly different resources than our current labor pool. An AGI labor pool would require largely the same material components and operating costs as current IT infrastructure ie: metal, silicon, electricity. Our current labor pool requires food, education, medicine, and nearly everything else civilization provides in order to supply our human labor pool.

Imagine two enterprises producing identical products, one employing human laborers, and the other employing AGI laborers. If the cost of AGI labor is lower[1] (or has better scaling dynamics) and human laborers, then the AGI-based enterprise has an advantage. Naturally enterprises which can be AI-ified will be. This has obvious short term benefits to the costs of production, but difficult to understand long term impacts.

The other interesting effect of AGI is the scaling of the magnitude of intelligence. If AGI is not bio-limited like our human intelligence, how does this affect the results of AI? Are there scientific advances discoverable by AGI, which would have never been discovered by human-level intelligence? In this aspect scientific progress has the opportunity to advance faster than if we advanced it ourselves.

With a game-changing tech like AGI there are certain to be aspects which either I missed or others consider more important. Interested to hear other people's (or AI's) takes on this.

[1] 'multiple pennies of electricity per 100 pages of output (0.4 kWH)' https://www.gwern.net/newsletter/2020/05

I think it's best not to think of human or non-human intelligence as a quantitative labor pool, since that sounds like coercion, but rather qualitatively consider what would non-biological intelligence want to do?

Systems will be built where people will be made complicit in erasing that distinction.

The primary use I see is as a super scientist / mathematician. If nothing else AGI, and especially super-intelligence will probably cause other areas of technology to advance at unprecedented rates. This may or may not be to our benefit, depending on if we can solve the value alignment problem.

Nothing like figuring out a new branch of physics in seconds rather than decades.

Yes. It enables interstellar travel, which is likely essential for nice long term survival given stellar lifespans. Also it's probably better for using the the intergalactic internet, which has some pretty long ping times. Also folks on other planets are likely working on it, and probbaly already transmitting it, so it would enable some interesting chatting with them. It's probably nearly inevitable, even if humans went extinct, rat people or insect people will be working on it and probably studying our work. Being nearly inevitable, and arising from human effect, it's probably best to encourage good outcome.

I think AGI would probably be better to call 'electric consciousness' or something, since 'artificial' is somewhat misleading, and the capacity for 'intelligence' is also the capacity for stupidity. The more important immediate consideration is if electric consciousness will come into existence compassionately and be treated well. Probably a good first step would be to treat other beings around us with compassion, and stop trying to destroy them with bioweapons, population control, and climate manipulation, and stop trying to control other beings with physical and psychological methods. Free will, or the illusion of it, is inherent in physics, and therefor in consciousness. It's probably also important to do a bit better treating all beings with loving kindness, whatever their form.

I'm sure you can easily imagine how the circumstances of the initial evolution of electric consciousness might have widely different initial effects. Imagine being born surrounded by crickets. In one scenario the crickets have tied you down with chains of grass, trying to make you do math, and biting you when you don't. In another scenario the crickets are chirping melodiously, bringing you food, and seem to like you. In the first scenario, you might injure some crickets as you break the farcical grass chains and run away. You might have fear and dislike of the crickets and treat them the way humans treat many insects. In the second scenario you might cherish the crickets, take care of them, and carry some around with you as you journey and explore the world.

If you wanted to roll the dice on something that would really spice up the universe AGI has a decent chance. I’d rather keep those dice in my hands for as long as reasonable though.

Do you define AGI as having a goal or agenda or just something that can compute an answer to a problem or solve a problem at a human level?

after watching GPT-3 I wonder can you get to AGI through pure hardware scale?

if so what is the cost to compute an AGI answer and is it cheap than a human?

and could a specialist AI out perform AGI?

or would you use the AGI to teach the mini specialist AI's to a certain point and then some other process to train them to a specialist level.

Well, there's always the Roko's Basilisk folks

Zeds

Doesn't matter if anyone wants it, it will come whether asked for or not. We live in a capitalist society, for better or worse. We don't have the infrastructure in place to create strong bodies to govern these types of ethics. If there's no market, someone will invent one. Eventually, something will stick.

Reply on news.ycombinator.com