Hacker News Reader

Comment by jorblumesea | Hacker News Reader

> Cracking AGI is a very long-term goal.

Is this fair to say? I feel like advancements in the field happened far quicker than anyone expected, and every few years we are reevaluating timelines. Especially given the research happening in tandem that will almost certainly speed up AGI, like graphene, quantum computing, advanced GPU design...

If you asked someone 10 years ago about the possibilities of ML/Deep learning they'd say it was far off too. I'm not going to say Kurzweil is correct but if I know anything, it's that historically these things have happened faster than expected. Look at 1997 -> 2017. 20 years, but what isn't changed?

Appreciate any discussion as I am not an expert :)

View on HN

We don't make specific time predictions because it's just not possible. But we can make relative predictions. However long it takes to get to AGI, when we're (say) half way there, the amount of societal upheaval will be enormous.

If we can't navigate that successfully, we'll never get to see AGI...

> when we're (say) half way there, the amount of societal upheaval will be enormous.

I had a conversation with a C level exec of a large company last week around this theme. My suggestion that limited AI such as self driving cars has the potential to create a vast number of extremely frustrated individuals making a second round of 'Sabotage' and Luddites a definite possibility was waved away as if those people don't matter and don't have any power.

I really wonder how far you'd have to be removed from the life of a truck or cab driver not to be able to empathize with them and to realize that if you take away some of the last low education jobs that allow you to keep your hands clean that there will be some kind of backlash.

AGI will expand that feeling to all of us.

If self driving cars/trucks become a thing, and cause millions of people to be unemployed, there is ALSO a correspondingly massive economy increase.

IE, the world is now massively richer because of all this awesome new technology.

Yes, there could be some short term disruption, but honestly I think things will end up fine, just because of the massive amount of extra money and wealth that the world will have that could be used to solve all those short term problems that disruption causes.

That's not how things have generally worked out in the world. Billions live in abject poverty even although the world has plenty enough to feed them and provide shelter.

The issue is income distribution - and just making more money doesn't magically fix the problem.

> just making more money doesn't magically fix the problem

Magically, no. With an effort yes. There are already experiments with basic pay for example.

Jeremy Rifkin books such as https://www.amazon.com/End-Work-Decline-Global-Post-Market/d... & 'Zero Marginal Cost' discuss the topic.

Yes exactly. We really hope to see this effort made. It's a fixable problem, but often in history great inequality is only fixed by violent revolution, which I don't want to see happen in my (or my daughter's) time!

Chances are that the increased riches of the world will not be divided in such a way that those who are now earning a living wage with their driving license will profit from that.

But it seems that increased economic benefits aren't shared. Thomas Piketty's Capital in the Twenty-First Century, average adjusted worker wages in the US from ~1975 to present, etc.

I'm concerned that 'some short term disruption' will actually be quite widespread and long-lasting, given increasing interdependencies.

You seem to forget why the luddites were luddities in the first place.

Yes, fabric was much cheaper, but in the meantime they were starving to death. The groups that make billions off this technology will lobby to keep more of their earnings so this vast wealth redistribution you want to happen, wont.

> However long it takes to get to AGI, when we're (say) half way there, the amount of societal upheaval will be enormous.

Could you unpack this a bit more?

Are you referring to the disruption in jobs caused by the effectiveness of narrow AI when we reach that point?

Yes exactly. Fewer people can add economic value at that point, and unless there is a negative income tax or basic guaranteed income, inequality spirals out of control, leading to even more of the anger-driven policy we're starting to see today.

We'll figure it out. Those who stand to benefit the most from AGI will want to keep their vast fortunes and lobby governments to figure out ways to keep the common man happy. Not to have every dream fulfilled, but just happy enough to not uprise. It's kinda always worked that way.

My opinion is that it is a very long-term goal. Compare it to the autonomous car, getting the first 90% of a complete autonomous car is much easier than the last 10%. With AGI probably it is the same. Deep learning has got traction the last years, but we are almost using the same algorithms than 20 years ago, just in better hardware (plus some tweaks to make them work with bigger models). But I think we know almost nothing about how to achieve AGI. Just compare our deep learning models with the models we have from our brains, they are a subset of what we know about our brains. And I don't think we know a lot about our brains, but very little.

Agreed. To date, Deep Learning has accomplished outstanding levels of discrimination and classification by combining supervised learning with probabilistic pattern matching. But it's widely acknowledged that most human learning is unsupervised and relies heavily on building upon large hierarchies of facts, their interdependencies, and a deep reliance on causation, none of which DL has yet shown any real facility toward realizing. One-shot examples of one-shot learning doth not a facile brain make.

Until DL shows significant progress toward these -- building knowledgebases and enabling their reuse -- and does so at the superlinear rate of development it's achieved for pattern recognition, I think it's likely that further development of the missing components of AGI will remain gradual and decades away.

You are right to say

> If you asked someone 10 years ago about the possibilities of ML/Deep learning they'd say it was far off too.

But when you read a little more about recent advances, you gain a healthy respect for the difficulty of the problems that are still left to solve. That tends to make researchers have lower expectations for an easy emergence of AGI than the general public.

We can't even generate a whole page of text that doesn't sound silly, with any neural network or AI algorithm to date. We're a long way off.

I don't know. It sounds more like the current approaches don't get us AGI. The machine learning tools are not enough. But that doesn't mean AGI is intrinsically hard. Maybe you just need a couple of different things in tandem, like OpenCog does.

I believe it is unfair (and unwise?) to throw in modifiers such as "very long-term" in the quoted sentence, or "very distant" in:

> It is hard for me to empathize with Musk’s fixation on evil super-intelligent AGI killer robots in a very distant future.

The truth is we don't know, (we can't know,) and we (on the whole) are making enormous strides. And it could take one small breakthrough to find ourselves "very" suddenly on the precipice of AGI and evil super-intelligent killer robots. Dismissing the current state of progress as trying to beat Atari is silly.

I, for one, am glad that some people with the resources are taking some efforts to get ahead of that "evil" aspect. But I don't see this as an "us" vs "them" scenario: it's also great that there are people helping make neural net algorithms more accessible.

My issue is that, while of course you're right that one can't know for sure, some people conclude that simply not being able to know justifies whipping up a frenzy and (in all likelihood) wasting part of a billion dollars that could go to, say, more productive research even within AI/ML.

Anyone could make a long list of things it's impossible to know for sure that would destroy humnaity. Many such things are vastly more likely to do so than malicious AGI.

Ok, I'll bite. Can you list several things that are vastly more likely to lead to the complete extinction of the human race (I'll assume that's what you meant by "destroy humanity") than malicious AGI?

Climate Change, leading to frequent crop failures, leading to the collapse of most nations and most industrial capacity.

That's what keeps me up at night these days.

That would be at the top of my list. And then there's nuclear attacks/disaster, asteroid hitting the earth, superbugs and biological weapons gone awry, physics experiment gone awry creating a black hole... I put superintelligent AI down towards alien invasion on my personal list of humanity-ending risks.

We rank all these scenarios very differently then :)

Climate change and nuclear war are huge issues, and we correctly invest orders of magnitude more money and time (which might still not be enough) trying to prevent or limit these than trying to prevent the birth of a malicious AGI, but they are extremely unlikely to lead to a complete extinction of the human race.

Regarding the other scenarios, if you allow me to move the goalposts from pure risk assessment:

- Preparing against an alien invasion seems futile, since given the timescales at play in the universe, the first aliens we meet will likely be millions of years behind or ahead of us.

- One way to survive an asteroid impact is to colonize another planet ahead of time, which Musk is working on.

- Physics experiments requiring specialized hardware like the LHC already have much more oversight than AI research where a breakthough could potentially happen in a garage on commodity hardware.

So it makes a lot of sense to me to invest some money in preventing an AI doomsday scenario next.

Musk says AI is the single biggest existential threat (unless I'm mistaken and someone else similarly famous said that, maybe Thiel?). From this it appears that you, along with the rest of society (as you mentioned by how much is spent on oversight), think he is wrong.

Nobody is arguing against funding AI. It's the fear-mongering we disagree with. It harms the field.

The question was about complete extinction. A population reduction by 90% is a horrible disaster, but it sets us back "just" a couple centuries, but it's hardly something that can eliminate humanity permanently; unlike quite a few other things.

- Asteroid strike

- Superbug pandemic

- Supervolcano eruption

Reply on news.ycombinator.com